IN32C:
New Approaches to Analyze Big Geoscientific Data Sets

Submit an Abstract to this Session

eLightning Session: This session will only be in eLightning format, an exciting new format that allows interactive, multimedia, electronic presentations and in-depth discussions. Learn more


Session ID#: 26340

Session Description:
Rapid analysis and interpretation of large model and measurement datasets is increasingly undertaken as a sequence of institution-supported pre-processing and user-devised post-processing (e.g., scripting of specialized statistics and visualization). By providing a pre-agreed format for data and metadata, the first stage ensures dataset utility and interoperability. In the second stage the user community employs diverse software practices and specialized toolkits to pursue their data analysis. Users now routinely attempt to ingest entire satellite records or MIP archives to complete their analysis. This often requires interactive and batch workflows to scale from the desktop to distributed-HPC systems. Such workflows must adjust to available memory constraints, provide access to CPU and cluster-level parallelism, while remaining flexible and easy to customize. How ought researchers utilize the unprecedented volume of data with metadata-aware analysis tools to answer tomorrow's data-intensive questions? This session will demonstrate state-of-the-art approaches to gigabyte- through petabyte-scale geoscientific data analysis.
Primary Convener:  Charles S Zender, University of California Irvine, Department of Earth System Science and Computer Science, Irvine, CA, United States
Conveners:  Joseph Hamman, National Center for Atmospheric Research, Hydrometeorological Applications Program, Research Applications Laboratory, Boulder, CO, United States and Ryan Abernathey, Columbia University of New York, Palisades, NY, United States
Chairs:  Joseph Hamman, National Center for Atmospheric Research, Hydrometeorological Applications Program, Research Applications Laboratory, Boulder, CO, United States, Charles S Zender, University of California Irvine, Department of Earth System Science and Computer Science, Irvine, CA, United States and Ryan Abernathey, Columbia University of New York, Palisades, NY, United States
OSPA Liaison:  Charles S Zender, University of California Irvine, Department of Earth System Science and Computer Science, Irvine, CA, United States

Cross-Listed:
  • A - Atmospheric Sciences
  • GC - Global Environmental Change
  • H - Hydrology
  • OS - Ocean Sciences
Index Terms:

0520 Data analysis: algorithms and implementation [COMPUTATIONAL GEOPHYSICS]
1932 High-performance computing [INFORMATICS]
1976 Software tools and services [INFORMATICS]
1998 Workflow [INFORMATICS]

Abstracts Submitted to this Session:

Large Scale Analysis of Geospatial Data with Dask and XArray (Invited) (214250)
Matthew Rocklin, Anaconda, Inc., Austin, TX, United States
Building confidence and credibility amid growing model and computing complexity (Invited) (207997)
Katherine J Evans, Oak Ridge National Laboratory, Oak Ridge, United States, Salil Mahajan, Oak Ridge Nat'l Lab, Oak Ridge, TN, United States, Milena Veneziani, Los Alamos National Laboratory, Los Alamos, NM, United States and Joseph H Kennedy, Oak Ridge National Laboratory, Oak Ridge, TN, United States
A Platform for Scalable Satellite and Geospatial Data Analysis (292009)
Carly Beneke1, Samuel Skillman1, Michael S Warren2, Tim Kelton1, Steven Patrick Brumby3, Rick Chartrand1 and Mark Mathis4, (1)Descartes Labs, Inc., Los Alamos, NM, United States, (2)None, College Station, TX, United States, (3)World Resources Institute, Washington, DC, United States, (4)Impact Observatory, Washington, DC, United States
Sifting Through SDO’s AIA Cosmic Ray Hits to Find Treasure (232822)
Michael S Kirk, Atmospheric and Space Technology Research Associates, LLC, Boulder, United States, Barbara J Thompson, NASA Goddard Space Flight Center, Greenbelt, MD, United States, Nicholeen M Viall, NASA GSFC, Greenbelt, United States and Peter R Young, George Mason University Fairfax, Fairfax, United States

See more of: Earth and Space Science Informatics