Scientific Data Management and Analysis for Earth Science at Scale

Abstract:

In a talk he delivered in 2007, Turing Award winner Jim Gray challenged computer scientists and systems builders; "We have to do better at producing tools to support the whole [scientific] research cycle—from data capture and data curation to data analysis and data visualization." He had been working closely with the Johns Hopkins team creating the Sloan Digital Sky Survey, and was struck by the disconnects between scientific data management requirements, and the collective orthodoxies of the relational (SQL) world. The SciDB open-source data management and analysis platform is our response to that challenge. In this talk, we will briefly review the state of play within the SciDB project: features, functionality and usage patterns. Then we will exploreseveral active earth science use cases at NASA, INPE and from the commercial sphere that use SciDB.