Improving data discovery and usability through commentary and user feedback: the CHARMe project

Abstract:

Earth science datasets are highly diverse. Users of these datasets are similarly varied, ranging from research scientists through industrial users to government decision- and policy-makers. It is very important for these users to understand the applicability of any dataset to their particular problem so that they can select the most appropriate data sources for their needs. Although data providers often provide rich supporting information in the form of metadata, typically this information does not include community usage information that can help other users judge fitness-for-purpose.

The CHARMe project (http://www.charme.org.uk) is filling this gap by developing a system for sharing "commentary metadata". These are annotations that are generated and shared by the user community and include:

• Links between publications and datasets. The CHARMe system can record information about why a particular dataset was used (e.g. the paper may describe the dataset, it may use the dataset as a source, or it may be publishing results of a dataset assessment). These publications may appear in the peer-reviewed literature, or may be technical reports, websites or blog posts.
• Free-text comments supplied by the user.
• Provenance information, including links between datasets and descriptions of processing algorithms and sensors.
• External events that may affect data quality (e.g. large volcanic eruptions or El Niño events); we call these "significant events".
• Data quality information, e.g. system maturity indices.

Commentary information can be linked to anything that can be uniquely identified (e.g. a dataset with a DOI or a persistent web address). It is also possible to associate commentary with particular subsets of datasets, for example to highlight an issue that is confined to a particular geographic region. We will demonstrate tools that show these capabilities in action, showing how users can apply commentary information during data discovery, visualization and analysis.

The CHARMe project has implemented a set of open-source tools to create, store and explore commentary information, using open Web standards. In this presentation we will describe the application of the CHARMe system to the particular case of the climate data community; however the techniques and technologies are generic and can be applied in many fields.