Supporting Research using Satellite Data: A Framework for Spatiotemporal Queries in SciDB

Abstract:

Natural phenomena such as haze, hurricane, and blizzard that evolve over time usually do not have well-defined boundaries. Their features may be captured by multiple satellites. To process and extract information from the large-scale satellite data, one needs a data-intensive architecture for distributed storage and computation resources. Such architecture allows end users such as scientists to effectively run their computation tasks with sharing computational resources and intermediate results, but without data replication.

The satellite data is most conveniently represented using arrays, exploiting its multidimensional nature. For our investigation, we use the open-source distributed, array-based SciDB as a platform for our spatiotemporal framework. SciDB conforms with the data-intensive architecture, providing a highly effectively computational and data storage platform. Moreover, it provides standard extension points, i.e., user defined data types, operators and functions.

Our current work focuses on more sophisticated indices including cartesian-coordinate indices, hierarchical triangular mesh and hybrid indices with data statistics and indexing. Furthermore, we introduce a spatiotemporal framework that allows us to generate and maintain indices according to given criteria and perform spatial and temporal operators and predicates. This framework overcomes weaknesses in SciDB where standard underlying array operations are less effective. We will demonstrate some examples (e.g., hurricane research using satellite data) of the functionalities in the proposed spatiotemporal framework.