A Survey of Best Practices for Long-Term Storage and Service of Data

Abstract:

NASA’s Heliophysics Data Management Policy calls for discipline-specific “final archives,” which will be responsible for the long-term archiving and service of Heliophysics mission data. Long-term archival functions, such as periodic revalidation of the data and migration to newer storage media when appropriate, have never been part of the Solar Data Analysis Center core capabilities. In addition, we recognize that the largest space solar physics data set, the SDO AIA and HMI data at the Stanford Joint Science and Operations Center (JSOC), will also need preservation and long-term access, as will the potentially much larger data archive of DKIST observations. We have carried out a study of data archiving best practices in other disciplines and organizations, including NASA’s Space Physics Data Facility (SPDF), the National Institute of Standards and Technology (NIST), and private industry, and report on the lessons learned and possible cost models. We seek input from the broader solar physics community on the relative value of various levels of preservation effort.