IN015:
Compressing Scientific Data: Reducing Storage While Preserving Information





Session ID#: 22360

Session Description:
Due to rapid technological progress our ability to generate increasingly larger data sets from high resolution numerical models is outpacing our ability to store, manage and effectively access these vast volumes of data. Similar statements can be made with regard to observational data captured by a variety of advanced instruments. One potential solution to this Big Data dilemma is the use of compression. Lossless compression offers perfect reconstruction, but provides only limited compaction when confronted with floating point data. Lossy compression, however, is able to achieve substantial reduction, but by its very definition is unable to exactly reproduce original values. The challenge then becomes addressing the question: how much information loss may be tolerated without affecting the interpretation of results, and how can this best be achieved? This session will address a variety of scientific data compression topics including: novel technologies, applications, and methods for evaluation.
Primary Convener:  John Clyne, NCAR, Boulder, CO, United States
Conveners:  Dorit Hammerling, National Center for Atmospheric Research, Institute for Mathematics Applied to Geosciences, Boulder, CO, United States and Allison H Baker, National Center for Atmospheric Research, Boulder, CO, United States

Cross-Listed:
  • A - Atmospheric Sciences
  • DI - Study of the Earth's Deep Interior
  • GC - Global Environmental Change
  • OS - Ocean Sciences
Index Terms:

0525 Data management [COMPUTATIONAL GEOPHYSICS]
0545 Modeling [COMPUTATIONAL GEOPHYSICS]
0550 Model verification and validation [COMPUTATIONAL GEOPHYSICS]
0599 General or miscellaneous [COMPUTATIONAL GEOPHYSICS]

Abstracts Submitted to this Session:

Max Zeyen1, James Ahrens1, Hans Hagen2, Katrin Heitmann3 and Salman Habib3, (1)Los Alamos National Laboratory, Los Alamos, NM, United States, (2)University of Kaiserslautern, Kaiserslautern, Germany, (3)Argonne National Laboratory, Argonne, IL, United States
Lin Xiong1, Guoquan Wang1 and Paul Wessel2, (1)University of Houston, Houston, TX, United States, (2)Univ Hawaii, Honolulu, HI, United States
Felipe Tagle, University of Notre Dame, Applied and Computational Mathematics and Statistics, Notre Dame, IN, United States, Stefano Castruccio, University of Notre Dame, Notre Dame, IN, United States, Paola Crippa, University of Notre Dame, Department of Civil & Environmental Engineering & Earth Sciences, Notre Dame, United States and Marc Genton, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia
Haiying Xu1, Allison Baker1, Dorit Hammerling2, Shaomeng Li1 and John Clyne3, (1)National Center for Atmospheric Research, Boulder, CO, United States, (2)National Center for Atmospheric Research, Institute for Mathematics Applied to Geosciences, Boulder, CO, United States, (3)NCAR, Boulder, CO, United States
Joseph Guinness, North Carolina State University Raleigh, Raleigh, NC, United States and Dorit Hammerling, National Center for Atmospheric Research, Institute for Mathematics Applied to Geosciences, Boulder, CO, United States
Jacob Tomlinson, Rachel Prudden, Niall Robinson and Alberto Arribas, Met Office, Informatics Lab, Exeter, United Kingdom
Charles S Zender, University of California Irvine, Departments of Earth System Science and Computer Science, Irvine, CA, United States and Jeremy Silver, The University of Melbourne, Melbourne, Australia
Peter Lindstrom, Lawrence Livermore National Laboratory, Livermore, CA, United States
Leigh Orf, Cooperative Institute for Meteorological Satellite Studies, Madison, WI, United States