A new Ensemble Consistency Test for the Community Earth System Model

Abstract:

Climate simulations codes, such as the Community Earth System Model (CESM), are especially complex and continually evolving. Their on-going state of development requires frequent software verification in the form of quality assurance to both preserve the quality of the code and instill model confidence. To formalize and simplify this previously subjective and computationally-expensive aspect of the verification process, we have developed a new tool for evaluating climate consistency. Because an ensemble of simulations allows us to gauge the natural variability of the model's climate, our new tool uses an ensemble approach for consistency testing. As many of the variables are highly correlated, we opt for a Principal Component Testing setup, which projects the original variables unto orthogonal linear combinations. Using the ensemble, we obtain a statistical distribution that can be used to determine whether a new climate run is statistically distinguishable from the original ensemble.

An important question is how to create the ensemble and what ensemble size to use. We tested perturbation in initial conditions and using different compilers to create the ensemble and developed a systematic approach to assess the required ensemble size. The CESM Ensemble Consistency Test has already proven its utility in detecting errors in software and hardware environments and providing rapid feedback to model developers. The test is now incorporated with new CESM releases for the atmosphere part of the model. We present results from theoretical and practical cases studies and an extension of the testing framework to the ocean model.