Decomposition of Model Forecast Errors: Methodology and Application

Abstract:

Root-Mean-Square Error (RMSE) has long been used as a performance metric for evaluating climate and weather forecast models. In this presentation it will be shown analytically that RMSE at times misrepresents model performance. A decomposition technique is proposed to describe more precisely RMSE distributions. Conventional RMSE can be decomposed into Error of Mean Difference (Em) and Error of Patter Variation (Ep). Ep is unbiased and can be used as an objective measure of model performance only if the anomalous pattern correlation (R) between forecast and analysis approaches to one. If R is small, Ep is biased and favors smoother forecasts that have smaller variances. Ep normalized by analysis variance is still biased and favors forecasts with smaller variance if anomalous pattern correlation is not perfect. A comprehensive evaluation of model errors should include Anomalous Pattern Correlation, Ratio of Forecast Variance to Analysis Variance, Error of Mean Difference, and Error of Pattern Variation. NCEP Global Forecast Systems with different configurations will be used to demonstrate the decomposition technique, which will be applied to both scalar variables and vector winds. At the end, the advantage and limitation of using data assimilation analysis increments to diagnose model biases will be discussed.