IN13A-1829
Agile based "Semi-"Automated Data ingest process : ORNL DAAC example
Monday, 14 December 2015
Poster Hall (Moscone South)
Suresh Kumar Santhana Vannan1, Tammy Beaty2, Robert B Cook1, Ranjeet Devarakonda1, Leslie Hook3, Yaxing Wei1 and Daine Wright1, (1)Oak Ridge National Laboratory, Oak Ridge, TN, United States, (2)ORNL, Oak Ridge, TN, United States, (3)Oak Ridge National Laboratory, Carbon Dioxide Information Analysis Center, Oak Ridge, TN, United States
Abstract:
The ORNL DAAC archives and publishes data and information relevant to biogeochemical, ecological, and environmental processes. The data archived at the ORNL DAAC must be well formatted, self-descriptive, and documented, as well as referenced in a peer-reviewed publication. The ORNL DAAC ingest team curates diverse data sets from multiple data providers simultaneously. To streamline the ingest process, the data set submission process at the ORNL DAAC has been recently updated to use an agile process and a semi-automated workflow system has been developed to provide a consistent data provider experience and to create a uniform data product. The goals of semi-automated agile ingest process are to: 1.Provide the ability to track a data set from acceptance to publication 2. Automate steps that can be automated to improve efficiencies and reduce redundancy 3.Update legacy ingest infrastructure 4.Provide a centralized system to manage the various aspects of ingest. This talk will cover the agile methodology, workflow, and tools developed through this system.
