Operational Flare Forecasting Benchmarks and Initial Performance Comparisons

It is the end of a magnetic cycle, and we recently asked two questions: (1) "How well do operational flare forecasting methods presently work?" and (2) "What is needed to quantitatively answer that question to begin with?" We present here select preliminary results from a recent workshop, "Benchmarks for Operational Solar Flare Forecasts" held at the Institute for Sun-Earth Environmental Research (ISEE) in Nagoya, Japan, in late 2017. Numerous methods were tested in a head-to-head operational forecasting performance exercise. Results are quantified using standard validation metrics, with a preference for metrics based on the probabilistic forecasts (rather than categorical results which are impacted by probability thresholds). We discuss how to best assess the relative performance of different methods, and present an initial analysis of general method attributes, addressing questions centered on ``which approaches lead to improvement in operational performance, and which approaches do not?''

Support for the workshop and this analysis is acknowledged from the Nagoya University/Institute for Space-Earth Environmental Research (ISEE) Center for International Collaborative Research (CICR).