Big Data for a Big Ocean at the NOAA National Oceanographic Data Center

Abstract:

Covering most of planet Earth, the vast, physically challenging ocean environment was once the sole domain of hardy, sea-going oceanographers. More recently, however, ocean observing systems have become more operational as well as more diverse. With observations coming from satellites, automated ship-based systems, autonomous underwater and airborne vehicles, in situ observing systems, and numerical models the field of oceanography is now clearly in the domain of Big Data. The NOAA National Oceanographic Data Center (NODC) and its partners around the world are addressing the entire range of Big Data issues for the ocean environment. A growing variety, volume, and velocity of incoming “Big Ocean” data streams are being managed through numerous approaches including the automated ingest and archive of incoming data; deployment of standardized, machine-consumable data discovery services; and interoperable data access, visualization, and subset mechanisms. In addition, support to the community of data producers to help them create more machine-ready ocean observation data streams is being provided and pilot projects to effectively incorporate commercial and hybrid cloud storage, access, and processing services into existing workflows and systems are being conducted. NODC is also engaging more actively than ever in the broader community of environmental data facilities to address these challenges. Details on these efforts at NODC and its partners will be provided and input sought on new and evolving user requirements.