Evaluating Consistency in the Ocean Model Component of the Community Earth System Model

Abstract:

We developed a new ensemble-based statistical method for evaluating the consistency in the Parallel Ocean Program (POP), the ocean model component of the Community Earth System Model (CESM). Since the ocean dynamics are chaotic in nature, a roundoff-level perturbation in the initial conditions will potentially result in distinct model solutions. No bit-for-bit (BFB) identical results in ocean solutions can be guaranteed for even tiny code modification. Our approach takes the natural variability of the ocean model into account through POP ensemble simulations. In particular, the statistical distribution from an ensemble of POP simulations is used to determine the standard score of any new model solution at each grid point. This setup accounts for the spatial heterogeneity in variability within the ensemble. Then the percentage of grid points which have scores greater than a specified threshold indicates whether the new model simulation is statistically distinguishable from the ensemble simulations. We evaluate the new tool on three types of scenarios: running with different processor layouts, changing the physical parameterization, and varying the convergence tolerance in the barotropic solver. Results indicate that our new testing tool is capable of distinguishing cases which should be consistent with the ensemble, such as the solutions with different processor layouts, and those which should not, such as increasing a certain physical parameter by two or more times. This new tool provides a simple, subjective and systematic way to evaluate the difference between the given solution and the ensemble, thus facilitating the detection of errors introduced during model development.