The M3 competition: Statistical tests of the results

By A. Koning, Philip Hans Franses, M. Hibon, Herman O. Stekler. 2005

203 citations recorded by the source corpus.

Subfields: Economics and Econometrics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Management Science and Operations Research

Citations in the corpus

Neural network ensemble operators for time series forecasting (2014, methodology): “In order to assess whether these diﬀerences are signiﬁcant or not, we employ the testing methodology suggested by Koning et al. (2005) that is appropriate for comparing forecasts from multiple models.”
The M4 competition: Bigger. Stronger. Better (2020, methodology): “First, we invited the best pool of methods whose performance did not statistically differ in terms of Multiple Comparisons from the Best (MCB; see Koning et al., 2005).”
Improving forecasting by estimating time series structural components across multiple frequencies (2014, methodology): “Across all datasets, both MAPA models are significantly better than the benchmark ETS for the medium, long and overall forecast horizons, as indicated by a Friedman followed by an MCB post-hoc test, according to the procedure outlined by Koning et al. (2005).”
Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction (2011, methodology): “…within the forecasting community (Armstrong, 2007a,b; Goodwin, 2007), we computed two non-parametric tests, replicating the analysis of the M3 by Koning et al. (2005): ANOM and MCB, both of which are based upon the average ranks of 41 methods (including both CI ensembles) over 111 series and 18…”
Significance Tests Harm Progress in Forecasting (2007, unknown): “Koning, Franses, Hibon and Stekler (2005) , referred to hereafter as KFHS, used tests of statistical”
An Empirical Comparison of Machine Learning Models for Time Series Forecasting (2010, methodology): “Further details of this test can be found in [26].”
Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation (2016, methodology): “In the M3 forecasting competition (Makridakis and Hibon, 2000; Koning et al., 2005), exponential smoothing methods obtained competitive results, and with the forecast package (Hyndman and Khandakar, 2008; Hyndman, 2014) in the programming language R (R Core Team, 2014), a fully automatic software…”
The inventory performance of forecasting methods: Evidence from the M3 competition data (2019, result): “In Figure 2, we also present the results from multiple comparisons from the best (MCB) test, as applied by Koning et al. (2005). Essentially, MCB tests if the average (across time series) rank of each method is significantly different from that of other methods.”