CMIP5 Model-analog Seasonal Forecast Skill: a Metric for Model Evaluation of ENSO

Ding et al. (2018, J. Climate) showed that tropical Indo-Pacific SST forecast skill from the North America Multi-Model Ensemble (NMME) can be matched or even exceeded using a simple "model-analog” method applied to existing long control runs from each NMME model. States taken directly from a long control run are chosen as model-analogs of each observed initial state over the years 1982-2015; then, their subsequent evolution within the control run provides a model-analog forecast ensemble. Subsequently, Ding et al. (2019, GRL) applied the model-analog method to preindustrial simulations from 28 different coupled general circulation model (CGCM) simulations in the Coupled Model Intercomparison Project (CMIP5) database. For most of the CMIP5 models, model-analogs provide skillful SST and precipitation hindcasts, with some as skillful as operational CGCMs during the post-1982 period.

Model-analog hindcast skill can also be used as a metric for ENSO simulations, since it evaluates how each CMIP5 model reproduces the observed evolution of ENSO over the historical record. As in Ding et al. (2018, 2019), model-analogs are determined within the tropical Indo-Pacific domain, using observed monthly SST and SSH anomalies. Analog ensembles corresponding to the observed anomalies are then identified in each of the 28 CMIP5 preindustrial control simulations. Retrospective forecasts of SST (1961-2015) and precipitation (1979-2015) are then made for leads of 1-12 months in the tropical Indo-Pacific. The long-term skill of the seasonal forecasts of tropical Pacific SST and precipitation represent how well each CMIP5 model’s attractor corresponds to nature’s attractor, in contrast with the more static aspects of ENSO measured by most existing metrics.

While precipitation in the tropical Pacific is not used to select the model-analogs, the precipitationforecast skill of the model-analogs emerges as a key identifier of the better tropical Indo-Pacific simulations. Several models with the best analog representation of observed SST anomalies actually display very poor precipitation forecast skill. The models with the highest precipitation forecast skill show the most realistic mean states for precipitation, SST, and surface winds in the equatorial Pacific. In addition, the most skillful models at predicting precipitation also better simulate the interannual variability of SST and precipitation in the equatorial Pacific. These results show a direct relationship between the mean model error and seasonal forecast error, which has been difficult to determine using more traditional forecast approaches. The potential for using model-analog forecast skill to identify models with more realistic 20^thcentury trends is also examined.