IN43D-3727:
Integrating Actionable User-defined Faceted Rules into the Hybrid Science Data System for Advanced Rapid Imaging & Analysis

Thursday, 18 December 2014
Gerald John Maramba Manipon1, Hook Hua1, Susan E Owen2, Gian Franco Sacco2, Piyush S. Agram2, Angelyn W Moore1, Sang-Ho Yun2, Eric Jameson Fielding3, Paul Lundgren1, Paul Alan Rosen1, Frank Webb4, Zhen Liu2, Alexander T Smith5, Brian D Wilson2, Mark Simons6, Michael P Poland7 and Peter F Cervelli8, (1)NASA Jet Propulsion Laboratory, Pasadena, CA, United States, (2)Jet Propulsion Laboratory, Pasadena, CA, United States, (3)Jet Propulsion Lab Caltech, Pasadena, CA, United States, (4)JPL, Pasadena, CA, United States, (5)NASA Jet Propulsion Laboratory, SIERRA MADRE, CA, United States, (6)California Institute of Technology, Pasadena, CA, United States, (7)Hawaiian Volcano Observatory, Hawaii National Park, HI, United States, (8)US Geological Survey, Menlo Park, CA, United States
Abstract:
The Hybrid Science Data System (HySDS) scalably powers the ingestion, metadata extraction, cataloging, high-volume data processing, and publication of the geodetic data products for the Advanced Rapid Imaging & Analysis for Monitoring Hazard (ARIA-MH) project at JPL. HySDS uses a heterogeneous set of worker nodes from private & public clouds as well as virtual & bare-metal machines to perform every aspect of the traditional science data system.

For our science data users, the forefront of HySDS is the facet search interface, FacetView, which allows them to browse, filter, and access the published products. Users are able to explore the collection of product metadata information and apply multiple filters to constrain the result set down to their particular interests. It allows them to download these faceted products for further analysis and generation of derived products. However, we have also employed a novel approach to faceting where it is also used to apply constraints for custom monitoring of products, system resources, and triggers for automated data processing.

The power of the facet search interface is well documented across various domains and its usefulness is rooted in the current state of existence of metadata. However, user needs usually extend beyond what is currently present in the data system. A user interested in synthetic aperture radar (SAR) data over Kilauea will download them from FacetView but would also want email notification of future incoming scenes. The user may even want that data pushed to a remote workstation for automated processing. Better still, these future products could trigger HySDS to run the user’s analysis on its array of worker nodes, on behalf of the user, and ingest the resulting derived products.

We will present our findings in integrating an ancillary, user-defined, system-driven processing system for HySDS that allows users to define faceted rules based on facet constraints and triggers actions when new SAR data products arrive that match the constraints. We will discuss use cases where users have defined rules for the automated generation of InSAR derived products: interferograms for California and Kilauea, time-series analyses, and damage proxy maps. These findings are relevant for science data system development of the proposed NASA-ISRO SAR mission.


Back to: Science Data System Architectures for New Earth Science Missions and Applications Posters
<< Previous Abstract | Next Abstract >>