Community-based benchmarking of the CMIP DECK experiments

Abstract:

A diversity of community-based efforts are independently developing “diagnostic packages” with little or no coordination between them. A short list of examples include NCAR’s Climate Variability Diagnostics Package (CVDP), ORNL’s International Land Model Benchmarking (ILAMB), LBNL’s Toolkit for Extreme Climate Analysis (TECA), PCMDI’s Metrics Package (PMP), the EU EMBRACE ESMValTool, the WGNE MJO diagnostics package, and CFMIP diagnostics. The full value of these efforts cannot be realized without some coordination. As a first step, a WCRP effort has initiated a catalog to document candidate packages that could potentially be applied in a “repeat-use” fashion to all simulations contributed to the CMIP DECK (Diagnostic, Evaluation and Characterization of Klima) experiments. Some coordination of community-based diagnostics has the additional potential to improve how CMIP modeling groups analyze their simulations during model-development. The fact that most modeling groups now maintain a “CMIP compliant” data stream means that in principal without much effort they could readily adopt a set of well organized diagnostic capabilities specifically designed to operate on CMIP DECK experiments. Ultimately, a detailed listing of and access to analysis codes that are demonstrated to work “out of the box” with CMIP data could enable model developers (and others) to select those codes they wish to implement in-house, potentially enabling more systematic evaluation during the model development process.