LaTiS: Interoperability via a Universal Functional Data Model

Abstract:

Datasets with disparate data formats and varied structural organization
require that scientific data users spend an inordinate amount of time
manipulating data instead of focusing on their area of expertise - the
science.

Numerous data models have been developed and used in various scientific
communities to ease the burden of data access and integration. However,
these data representations tend to be domain specific, making
interoperability across disciplines challenging.

We present an alternative approach towards modeling datasets that
takes a step closer to the mathematical structure of the dataset,
emphasizing the functional relationships that are inherent in most
scientific data. As a result, these Functional data structures enable a
more representative abstraction of the actual dataset than traditional data
models, facilitating higher level reasoning that is more closely related
to the problem domain. (For example, a "time series of spectra" as
opposed to multidimensional arrays.)

Since any dataset can be represented in this common mathematical form,
data access and integration difficulties can be greatly reduced. The
mathematical nature of the model also lends itself to an algebra that
can be used to manipulate datasets, further bringing disparate datasets
into alignment and simplifying analysis tasks.

LaTiS (https://github.com/dlindhol/LaTiS) is an Open Source
implementation of this Functional Data Model. In addition to the model
itself, the LaTiS software framework provides a pluggable architecture for
Writers that expose existing datasets via the model interface,
Operations that manipulate datasets in terms of the model, and Writers
that output a modeled dataset in a variety of forms. LaTiS has enabled
us to easily provide access to datasets in their native form and
transform them in real time to improve usability. With the LaTiS service
interface, a data provider can also stand up a server to provide
uniform access to their data.