Evaluation of Accuracy and Reliability of the Six Ensemble Methods Using 198 Sets of Pseudo-Simulation Data

Abstract:

The accuracy and reliability of the six ensemble methods were evaluated according to simulation skills, training period, and ensemble members, using 198 sets of pseudo-simulation data (PSD) generated by considering the simulation characteristics of regional climate models. The PSD sets were classified into 18 categories according to the relative magnitude of bias, variance ratio, and correlation coefficient, where each category had 11 sets with 50 samples. The ensemble methods used were as follows: equal weighted averaging with(out) bias correction (EWA_W(N)BC), weighted ensemble averaging based on root mean square errors and correlation (WEA_RAC), WEA based on the Taylor score (WEA_Tay), WEA based on reliability (WEA_REA), and multivariate linear regression (Mul_Reg). The weighted ensemble methods showed better projection skills in terms of accuracy and reliability than non-weighted methods, in particular, for the PSD categories having systematic biases and various correlation coefficients. In general, WEA_Tay, WEA_REA and WEA_RAC showed superior skills in terms of accuracy and reliability, regardless of the PSD categories, training periods, and ensemble numbers. The evaluation results showed that WEA_Tay and WEA_RAC are applicable even for simulation data with systematic biases, a short training period, and a small number of members. However, the EWA_NBC showed a comparable projection skill with the other methods only in the certain categories with unsystematic biases.