Evolving Data System Architectures in NOAA: Perspectives from the National Data Centers

Abstract:

For decades, the National Oceanic and Atmospheric Administration (NOAA) has operated three distinct National Data Centers to manage its large and diverse environmental data collections. These centers, the National Oceanographic Data Center (NODC), the National Geophysical Data Center (NGDC), and the National Climatic Data Center (NCDC), have collaborated over the years on various programs and projects to esnure the long term preservation and scientific stewardship of their archived data, workflows, and algorithms. In recent years, the pace of collaboration has accelerated dramatically as new observing missions have come online, as new designated communities have emerged, and as waves of consolidation have swept across NOAA, driven by technological, budgetary, and policy-oriented pressures. An update on how NODC, NGDC, and NCDC have responded to these pressures and have been evolving their data system architectures and operations to keep pace with the new requirements will be presented. Examples efforts in the areas of streamlined data ingest, improved data discoverability, and enhanced data interoperability will be provided to illustrate the Natonal Data Centers' committment to meeting the needs of their user communities and highlight the rapid evolution taking place in their science data systems.