HydroUnits: A Python-based Physical Units Management Tool in Hydrologic Computing Systems

Abstract:

While one objective of data management systems is to provide the units when annotating the collected data, another is that the units must be correctly manipulated during conversion steps. This is not a trivial task however and the units conversion time and errors for large datasets can be quite expensive. To date, more than a dozen Python modules have been developed to deal with units attached to quantities. However, they fall short in many ways and also suffer from not integrating with a units controlled vocabulary. Moreover, none of them permits the encoding of some complex units defined in the Consortium of Universities for the Advancement of Hydrologic Sciences, Inc.’s Observations Data Model (CUAHSI ODM) as a vectorial representation for storage demand reduction and does not incorporate provision to accommodate unforeseen standards-based units.

We developed HydroUnits, a Python-based units management tool for three specific purposes: encoding of physical units in the Transducer Electronic Data Sheet (TEDS) as defined in the IEEE 1451.0 standard, performing dimensional analysis and on-the-fly conversion of time series allowing users to retrieve data from a data source in a desired equivalent unit while accommodating unforeseen and user-defined units. HydroUnits differentiates itself to existing tools by a number of factors including the implementation approach adopted, the adoption of standard-based units naming conventions and more importantly the emphasis on units controlled vocabularies which are a critical aspect of units treatment. Additionally, HydroUnits supports unit conversion for quantities with additive scaling factor, and natively supports time series conversion and takes leap years into consideration for units consisting of the time dimension (e.g., month, minute). Due to its overall implementation approach, HydroUnits exhibits a high level of versatility that no other tool we are aware of has achieved.