The M4 Competition: 100,000 time series and 61 forecasting methods
By Spyros Makridakis, Evangelos Spiliotis, Vassilios Assimakopoulos. 2020
970 citations recorded by the source corpus.
Abstract The M4 Competition follows on from the three previous M competitions, the purpose of which was to learn from empirical evidence both how to improve the forecasting accuracy and how such learning could be used to advance the theory and practice of forecasting. The aim of M4 was to replicate and extend the three previous competitions by: (a) significantly increasing the number of series, (b) expanding the number of forecasting methods, and (c) including prediction intervals in the evaluation process as well as point forecasts. This paper covers all aspects of M4 in detail, including its organization and running, the presentation of its results, the top-performing methods overall and by categories, its major findings and their implications, and the computational requirements of the various methods. Finally, it summarizes its main conclusions and states the expectation that its series will become a testing ground for the evaluation of new methods and the improvement of the practice of forecasting, while also suggesting some ways forward for the field.
Subfields: Economics and Econometrics, Management Science and Operations Research, Evaluation & Benchmarks, Time-Series & Forecasting
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Economics and Econometrics
- Rob J Hyndman
- Fotios Petropoulos
- Konstantinos Nikolopoulos
- Nikolaos Kourentzes
- Vassilios Assimakopoulos
- Evangelos Spiliotis
- Spyros Makridakis
- Christoph Bergmeir
In Management Science and Operations Research
- Rob J Hyndman
- Fotios Petropoulos
- Konstantinos Nikolopoulos
- Nikolaos Kourentzes
- Vassilios Assimakopoulos
- Spyros Makridakis
- Evangelos Spiliotis
- Christoph Bergmeir
In Evaluation & Benchmarks
- Rob J Hyndman
- Fotios Petropoulos
- Konstantinos Nikolopoulos
- Nikolaos Kourentzes
- Christoph Bergmeir
- Vassilios Assimakopoulos
- Spyros Makridakis
- Evangelos Spiliotis
In Time-Series & Forecasting
- Fotios Petropoulos
- Rob J Hyndman
- Konstantinos Nikolopoulos
- Spyros Makridakis
- Nikolaos Kourentzes
- Christoph Bergmeir
- Ivan Svetunkov
- Evangelos Spiliotis
Citations in the corpus
- Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift (2022, methodology): “Ad-ditionally, we provide experimental results on three more datasets, including the air quality and Nas-daq datasets taken from the UCI repository and M4 competition dataset (Makridakis et al., 2020) in Appendix A.1.”
- Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions (2020, neutral): “For COVID-19, Petropoulos and Makridakis 2020) applied ETS ( Hyndman, Koehler, Snyder & Grose, 2002 ) odels for predicting the evolution of the number of cases at a lobal scale.”
- Unified Training of Universal Time Series Forecasting Transformers (2024, background): “Mean Scaled Interval Score The MSIS is a metric to evaluate uncertainty around point forecasts, introduced in the M4 Competition (Makridakis et al., 2020).”
- PDEBENCH: An Extensive Benchmark for Scientific Machine Learning (2022, background): “Various domains in machine learning have widely influential datasets: In time series forecasting there are the Makridakis competitions [36], in reinforcement learning there is the OpenAI Gym [5].”
- Forecasting: theory and practice (2020, methodology): “For COVID-19, Petropoulos and Makridakis (2020) applied a multiplicative exponential smoothing model for predicting global number of confirmed cases, with very successful results both for point forecasts and prediction intervals.”
- A simple combination of univariate models (2020, result): “An in-depth computational analysis of the SCUM approach compared to other submissions in the M4-competition is provided by Makridakis et al. (2019) in this special issue.”
- Forecasting the novel coronavirus COVID-19 (2020, background): “This family has shown good forecast accuracy over several forecasting competitions [7] [8] [9] and is especially suitable for short series.”
- Forecast combinations: An over 50-year review (2022, background): “However, the results from the recent M4 competition (Makridakis et al., 2020a) showed that simple combinations continue to achieve relatively good forecasting performance and are still competitive.”
- N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting (2022, positive): “The evident success of the methods in recent forecasting competitions (Makri-dakis, Spiliotis, and Assimakopoulos 2020, 2021) has reno-vated the interest within the academic community (Benidis et al. 2020 Multi-step forecasting.”