New Approaches to Analyze Big Geoscientific Datasets

eLightning Session: This session will only be in eLightning format, an exciting new format that allows interactive, multimedia, electronic presentations and in-depth discussions. Learn more

Session ID#: 26340

Session Description:
Rapid analysis and interpretation of large model and measurement datasets is increasingly undertaken as a sequence of institution-supported pre-processing and user-devised post-processing (e.g., scripting of specialized statistics and visualization). By providing a pre-agreed format for data and metadata, the first stage ensures dataset utility and interoperability. In the second stage the user community employs diverse software practices and specialized toolkits to pursue their data analysis. Users now routinely attempt to ingest entire satellite records or MIP archives to complete their analysis. This often requires interactive and batch workflows to scale from the desktop to distributed-HPC systems. Such workflows must adjust to available memory constraints, provide access to CPU and cluster-level parallelism, while remaining flexible and easy to customize. How ought researchers utilize the unprecedented volume of data with metadata-aware analysis tools to answer tomorrow's data-intensive questions? This session will demonstrate state-of-the-art approaches to gigabyte- through petabyte-scale geoscientific data analysis.
Primary Convener:  Charles S Zender, University of California Irvine, Departments of Earth System Science and Computer Science, Irvine, CA, United States
Conveners:  Joseph Hamman, National Center for Atmospheric Research, Hydrometeorological Applications Program, Research Applications Laboratory, Boulder, CO, United States and Ryan Abernathey, Columbia University of New York, Palisades, NY, United States

  • A - Atmospheric Sciences
  • GC - Global Environmental Change
  • H - Hydrology
  • OS - Ocean Sciences
Index Terms:

Abstracts Submitted to this Session:

Yuan Ho and Jeff Weber, University Corporation for Atmospheric Research, Boulder, CO, United States
Matthew Rocklin, Continuum Analytics, Austin, TX, United States
Catherine Olschanowsky, Alejandro N Flores, Katelyn FitzGerald, Matthew T. Masarik, William J Rudisill and Miguel Aguayo, Boise State University, Boise, ID, United States
Carly Marie Beneke1, Samuel Skillman1, Michael S Warren2, Tim Kelton3, Steven Patrick Brumby1, Rick Chartrand1 and Mark Mathis1, (1)Descartes Labs, Inc., Los Alamos, NM, United States, (2)College Station, TX, United States, (3)Gangseo-Gu Seoul, Korea, Republic of (South)
Justin Gerard Simcock1, Michael Delgado2, Michael Greenstone3, Solomon M Hsiang4, Robert E Kopp5, Tamma Carleton6, Andrew Hultgren7, Amir Jina8, Ishan Nath9, James A Rising10, Ashwin Rode9, Jiacan Yuan11, Trinetta Chong12, Greg Dobbels13, Azhar Hussain13, Yuqi Song13, Jingyuan Wang13, Shashank Mohan14, Kate Larsen14 and Trevor Houser14, (1)Oakland, CA, United States, (2)Rhodium Group, Oakland, United States, (3)University of Chicago, Chicago, IL, United States, (4)University of California Berkeley, Berkeley, CA, United States, (5)Rutgers University, Institute of Earth, Ocean and Atmospheric Sciences, New Brunswick, NJ, United States, (6)University of California, Berkeley, Berkeley, CA, United States, (7)University of California Berkeley, Agricultural and Resource Economics, Berkeley, CA, United States, (8)Columbia University of New York, Palisades, NY, United States, (9)University of Chicago, Economics, Chicago, IL, United States, (10)London School of Economics, Grantham Institute on Climate Change, London, United Kingdom, (11)Duke University, Earth and Ocean Sciences, Durham, NC, United States, (12)University of Chicago, Energy Policy Institute at Chicago, Chicago, IL, United States, (13)University of Chicago, Energy Policy Institute of Chicago, Chicago, IL, United States, (14)Rhodium Group, Oakland, CA, United States
Katherine J Evans1, Salil Mahajan2, Carmela Veneziani3 and Joseph H Kennedy1, (1)Oak Ridge National Laboratory, Oak Ridge, TN, United States, (2)Oak Ridge Nat'l Lab, Oak Ridge, TN, United States, (3)Los Alamos National Laboratory, Los Alamos, NM, United States