A Distributed, Open Source based Data Infrastructure for the Megacities Carbon Project

Abstract:

With the goal of assessing the anthropogenic carbon-emission impact of urban centers on local and global climates, the Megacities Carbon Project has been building carbon-monitoring capabilities for the past two years around the Los Angeles metropolitan area. Hundreds of megabytes (MB) of data are generated daily, and distributed among data centers local to the sensor networks involved. We automatically pull this remotely generated data into a centralized data infrastructure local to the Jet Propulsion Laboratory (JPL), seeking to (1) provide collaboration opportunities on the data, and (2) generate refined data products through community-requested centralized data processing pipelines. The goal of this informatics effort is to ensure near real-time access to generated data products across the Los Angeles carbon monitoring sensor network and meet the data analysis needs of carbon researchers through the production of customized products. We discuss the goals of the informatics effort, its uniqueness, and assess its effectiveness in providing an insight into the carbon sphere of Los Angeles.