Using OPeNDAP's Data-Services Framework to Lift Mash-Ups above Blind Dates

Abstract:

OPeNDAP's data-as-service framework (Hyrax) matches diverse sources with many end-user tools and contexts. Keys to its flexibility include:

• A data model embracing tabular data alongside n-dim arrays and other structures useful in geoinformatics.

• A REST-like protocol that supports—via suffix notation—a growing set of output forms (netCDF, XML, etc.) plus a query syntax for subsetting.

• Subsetting applies (via constraints on column values) to tabular data or (via constraints on indices or coordinates) to array-style data .

• A handler-style architecture that admits a growing set of input types. Community members may contribute handlers, making Hyrax effective as middleware, where N sources are mapped to M outputs with order N+M effort (not NxM).

• Hyrax offers virtual aggregations of source data, enabling granularity aimed at users, not data-collectors.

• OPeNDAP-access libraries exist in multiple languages, including Python, Java, and C++.

Recent enhancements are increasing this framework’s interoperability (i.e., its mash-up) potential.

• Extensions implemented as servlets—running adjacent to Hyrax—are enriching the forms of aggregation and enabling new protocols:

• User-specified aggregations, namely, applying a query to (huge) lists of source granules, and receiving one (large) table or zipped netCDF file.

• OGC (Open Geospatial Consortium) protocols, WMS and WCS.

• A Webification (W10n) protocol that returns JavaScript Object Notation (JSON).

• Extensions to OPeNDAP’s query language are reducing transfer volumes and enabling new forms of inspection. Advances underway include:

• Functions that, for triangular-mesh sources, return sub-meshes spec’d via geospatial bounding boxes.

• Functions that, for data from multiple, satellite-borne sensors (with differing orbits), select observations based on coincidence.

• Calculations of means, histograms, etc. that greatly reduce output volumes..

• Paths for communities to contribute new server functions (in Python, e.g.) that data providers may incorporate into Hyrax via installation parameters.

One could say Hyrax itself is a mash-up, but we suggest it as an instrument for a mash-up artist’s toolbox. This instrument can support mash-ups built on netCDF files, OGC protocols, JavaScript Web pages, and/or programs written in Python, Java, C or C++.