Evaluating TEC medium range forecast based on the True Skill Score approach

Abstract:

Evaluating medium range (1 to 4 days) total electron content (TEC) forecast is important for understanding and improving ionosphere-thermosphere (IT) forecasting capabilities. We focus on performance of three physics-based global IT models, GITM, TIEGCM and CTIPe. These models were run in a “forecast” mode at the Community Coordinated Modeling Center (CCMC) for a number of representative CME and HSS-type storms in 2000-2016. In the forecast mode, the simulations are performed under standard settings and are driven by forecastable solar wind inputs and forecasted F10.7 index. Global Ionospheric Maps (GIM) routinely produced by JPL are utilized to characterize ionospheric TEC for the simulated events. We extend the TEC metric approach by Meng et al. (2016) by implementing the True Skill Statistic (TSS, e.g., Bloomfield et al. (2012)). Forecasted TEC disturbances and GIM values were compared on a grid and occurrences of true positives, false positives and false negatives during ionospheric storms are analyzed. Further challenges in evaluating medium range TEC forecast are discussed. Robust metrics can help the community to better evaluate forecasts and establish best practices to be used across applications.

Bloomfield, D. S., P. A. Higgins, R. T. J. McAteer, and P. T. Gallagher (2012), Toward reliable benchmarking of solar flare forecasting methods, The Astrophysical Journal Letters, 747 (2), L41.

Meng, X., A. J. Mannucci, O. Verkhoglyadova, and B.T. Tsurutani (2016), On forecasting ionospheric total electron content responses to high-speed solar wind streams, J. Space Weather Space Clim., 6, A19,10.1051/swsc/2016014.