TOWARD RELIABLE BENCHMARKING OF SOLAR FLARE FORECASTING METHODS

By D. S. Bloomfield, P. Higgins, R. McAteer, Peter T. Gallagher. 2012

222 citations recorded by the source corpus.

Solar flares occur in complex sunspot groups, but it remains unclear how the probability of producing a flare of a given magnitude relates to the characteristics of the sunspot group. Here, we use Geostationary Operational Environmental Satellite X-ray flares and McIntosh group classifications from solar cycles 21 and 22 to calculate average flare rates for each McIntosh class and use these to determine Poisson probabilities for different flare magnitudes. Forecast verification measures are studied to find optimum thresholds to convert Poisson flare probabilities into yes/no predictions of cycle 23 flares. A case is presented to adopt the true skill statistic (TSS) as a standard for forecast comparison over the commonly used Heidke skill score (HSS). In predicting flares over 24 hr, the maximum values of TSS achieved are 0.44 (C-class), 0.53 (M-class), 0.74 (X-class), 0.54 (⩾M1.0), and 0.46 (⩾C1.0). The maximum values of HSS are 0.38 (C-class), 0.27 (M-class), 0.14 (X-class), 0.28 (⩾M1.0), and 0.41 (⩾C1.0). These show that Poisson probabilities perform comparably to some more complex prediction systems, but the overall inaccuracy highlights the problem with using average values to represent flaring rate distributions.

Subfields: Artificial Intelligence, Astronomy and Astrophysics

View Paper: https://www.semanticscholar.org/paper/a3c1b7750701c68fe34f69b1e2c45ffda939f7c5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Astronomy and Astrophysics

Citations in the corpus

Space weather: the solar perspective (2021, background): “…flares seems to be more easily achievable compared to flares of medium to low energy, however, the uncertainties coming from the statistical approach are rather large preventing from more accurate single event prediction (see e.g., Schrijver 2007; Georgoulis and Rust 2007; Bloomfield et al. 2012).”
The Challenge of Machine Learning in Space Weather: Nowcasting and Forecasting (2019, background): “For extensive discussions about metric selection, the reader is referred to Bloomfield et al. (2012), Bobra and Couvidat (2015), Liemohn et al. (2018), and Morley et al. (2018).”
A COMPARISON OF FLARE FORECASTING METHODS. I. RESULTS FROM THE “ALL-CLEAR” WORKSHOP (2016, methodology): “This can be particularly helpful when comparing studies performed on different data sets (Hanssen & Kuipers 1965; Bloomfield et al. 2012).”
SOLAR FLARE PREDICTION USING SDO/HMI VECTOR MAGNETIC FIELD DATA WITH A MACHINE-LEARNING ALGORITHM (2014, methodology): “Bloomfield et al. (2012) highlight in their Table 4 that including C-class flares may improve some performance metrics while lowering others: in their case, not including C-class flares increase the True Skill Statistic (TSS) metric but decrease the Heidke Skill Score (HSS) (both quantities are…”
Predicting Solar Flares Using SDO/HMI Vector Magnetic Data Products and the Random Forest Algorithm (2017, methodology): “By looking at the TSS score, it is apparent that the performance of our method coupling SDO/HMI magnetic parameters with RF (TSS ≈ 0.53) is very similar to Bloomfield et al. (2012) and Ahmed et al. (2013), while being ∼28% lower than Bobra & Couvidat (2015) and Nishizuka et al. (2017).”