IN045:
New Approaches to Analyze Big Geoscientific Datasets

Submit an Abstract to this Session

eLightning Session: This session will only be in eLightning format, an exciting new format that allows interactive, multimedia, electronic presentations and in-depth discussions. Learn more


Session ID#: 26340

Session Description:
Rapid analysis and interpretation of large model and measurement datasets is increasingly undertaken as a sequence of institution-supported pre-processing and user-devised post-processing (e.g., scripting of specialized statistics and visualization). By providing a pre-agreed format for data and metadata, the first stage ensures dataset utility and interoperability. In the second stage the user community employs diverse software practices and specialized toolkits to pursue their data analysis. Users now routinely attempt to ingest entire satellite records or MIP archives to complete their analysis. This often requires interactive and batch workflows to scale from the desktop to distributed-HPC systems. Such workflows must adjust to available memory constraints, provide access to CPU and cluster-level parallelism, while remaining flexible and easy to customize. How ought researchers utilize the unprecedented volume of data with metadata-aware analysis tools to answer tomorrow's data-intensive questions? This session will demonstrate state-of-the-art approaches to gigabyte- through petabyte-scale geoscientific data analysis.
Primary Convener:  Charles S Zender, University of California Irvine, Departments of Earth System Science and Computer Science, Irvine, CA, United States
Conveners:  Joseph Hamman, National Center for Atmospheric Research, Hydrometeorological Applications Program, Research Applications Laboratory, Boulder, CO, United States and Ryan Abernathey, Columbia University of New York, Palisades, NY, United States

Cross-Listed:
  • A - Atmospheric Sciences
  • GC - Global Environmental Change
  • H - Hydrology
  • OS - Ocean Sciences
Index Terms:

Abstracts Submitted to this Session:

Yuan Ho and Jeff Weber, University Corporation for Atmospheric Research, Boulder, CO, United States
Matthew Rocklin, Continuum Analytics, Austin, TX, United States
Katherine J Evans1, Salil Mahajan2, Carmela Veneziani3 and Joseph H Kennedy1, (1)Oak Ridge National Laboratory, Oak Ridge, TN, United States, (2)Oak Ridge Nat'l Lab, Oak Ridge, TN, United States, (3)Los Alamos National Laboratory, Los Alamos, NM, United States