Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons

By J. Armstrong, F. Collopy. 1992

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Neural networks for short-term load forecasting: a review and evaluation (2001, methodology): “The choice of error measures to help comparing forecasting methods has been much discussed, as a consequence of the many competitions that were started in the 1980’s [ 4 ], [5], [32].”
HydroTest: A web-based toolbox of evaluation metrics for the standardised assessment of hydrological forecasts (2007, background): “It records as a ratio the level of overall agreement between the observed and modelled datasets and is often expressed in percentage terms; the Mean Absolute Percent Error (MAPE; Armstrong and Collopy, 1992; Maier and Dandy, 1996; Bowden et al., 2002).”
Making and Evaluating Point Forecasts (2009, methodology): “…arise, such as the comparability of scores across time series with realizations of differing magnitude and volatility, and commonly used evaluation methods remain controversial (Armstrong and Collopy 1992; Fildes 1992; Ahlburg et al. 1992; Hyndman and Koehler 2006; Athanasopoulos et al. 2011).”
The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation (2021, unknown): “However, its has a number of drawbacks, too, the most critical ones being the restriction of its use to strictly positive data by de ﬁ nition and being biased towards low forecasts, which makes it unsuitable for predictive models where large errors are expected ( Armstrong & Collopy, 1992 ).”
Mean-based error measures for intermittent demand forecasting (2013, methodology): “It is used by [18], and [1] recommends the use of relative error-based measures.”
Golden Rule of Forecasting: Be conservative (2015, unknown): “Given the power of the no-change model in many situations, the Relative Absolute Error (RAE) was developed to compare the accuracy of forecasts from alternative models (Armstrong and Collopy, 1992).”
MAPE-R: a rescaled measure of accuracy for cross-sectional subnational population forecasts (2011, background): “Other measures are based on relative errors ( Armstrong and Collopy 1992;”
Prediction Intervals (1993, background): “…PMSE), namely E[e (h) ] (bu n 2 e p note that scale-independent statistics, such as the mean absolute prediction error (MAPE), will b referred for comparing the accuracy of forecasts made for different variables, especially when , E measured on different scales (Armstrong & Collopy 1992).”
Time prediction based on process mining (2011, background): “This is a known problem of this error measure [8], therefore, we do not report on MAPE values in the remainder.”
Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences (2020, unknown): “ Nonsymmetrical [49]. 20 predictions Normalized Root Mean Squared Error (NRMSE)** Normalizes the root mean squared error. �� = √∑ �� 2 �=1 � ��𝑒𝑎� •Can be used to compare from different datasets [55]. 21 Normalized Mean Squared Error (NMSE) Estimates the overall deviations between measured va”
25 Years of IIF Time Series Forecasting: A Selective Review (2005, methodology): “Note that Armstrong & Collopy (1992) referred to RelMAE as CumRAE, and that RelRMSE is also known as Thiel’s U statistic (Theil, 1966, Chapter 2) and is sometimes called U2.”
Another Look at Forecast Accuracy Metrics for Intermittent Demand (2006, background): “Then we can define Because they are not scale dependent, these relative-error metrics were recommended in studies by Armstrong and Collopy (1992) and by Fildes (1992) for assessing forecast accuracy across multiple series.”