Emerging Lessons from the NOAA/NESDIS Cloud Pilot Project

Building from the lessons learned from other cloud migration and pilot projects scattered around NOAA and the broader community, NOAA/NESDIS initiated an enterprise-scale effort in 2019 to establish its comprehensive data management framework in the cloud, forming the NESDIS Cloud Integrated Project Team and tasking it to deliver the functioning pilot capability by spring of 2020. The NESDIS cloud framework consists of a consolidated secure ingest capability, storage and catalog services capable of tracking billions of granules, a common data discovery API and UI, plus a science and development sandbox and a high performance compute environment. Security and analytics are embedded throughout the framework and progress toward a fully-automated Continuous Integration/Continuous Delivery pipeline for deploying product generation and other other applications has been achieved. The framework is being developed with flexibility in mind, to meet the needs of the entire NESDIS mission and possibly broader NOAA requirements as well. The current status of these efforts will be presented, with an emphasis on the lessons learned by NESDIS in enabling the cloud-based exploitation of NOAA’s ocean of big data.