Out-of-sample tests of forecasting accuracy: an analysis and review

By Len Tashman. 2000

879 citations recorded by the source corpus.

Subfields: Economics and Econometrics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Management Science and Operations Research

Citations in the corpus

Forecasting at Scale (2018, methodology): “This procedure is based on classical “rolling origin” forecast evaluation procedures (Tashman 2000), but uses only a small sequence of cutoﬀ dates rather than making one forecast per historical date.”
Long Term Probabilistic Load Forecasting and Normalization With Hourly Information (2014, methodology): “At the end, we compare the ex post forecasting accuracy of the C1, C2, B, S and L models on a rolling basis using 2007 through 2010 [15].”
Forecasting: theory and practice (2020, background): “To overcome these problems, the so-called time-series cross-validation (Hyndman and Athanasopoulos, 2018) extends the out-of-sample approach from a fixed origin to a rolling origin evaluation (Tashman, 2000).”
25 years of time series forecasting (2006, methodology): “Several software vendors have implemented automated time series forecasting methods (including multivariate methods); see, e.g., Geriner & Ord (1991), Tashman & Leach (1991) and Tashman (2000).”
Forecasting model selection through out-of-sample rolling horizon weighted errors (2011, background): “Overfitting and structural changes may further aggravate the divergence between in-sample and post-sample performance (Tashman, 2000).”
Neural network ensemble operators for time series forecasting (2014, background): “The reader is referred to Tashman (2000) for a detailed description of the evaluation scheme and its advantages.”
Evaluating time series forecasting models: an empirical study on performance estimation methods (2019, methodology): “rent strategies regarding training/testing split point, growing or sliding window settings, and eventual update of the models. In order to produce a robust estimate of predictive performance, Tashman [38] recommends employing these strategies in multiple test periods. One might create dierent sub-sa”
An evaluation of neural network ensembles and model selection for time series prediction (2010, background): “Furthermore, the error estimation is robust against irregular origins [ 21 ].”
Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction (2011, background): “…(and, where possible, a rolling origin design), the use of multiple robust error metrics, a comparison with established (statistical) benchmark algorithms, and the analysis of the data conditions under which a method performs well (Tashman, 2000), in order to obtain valid and reliable results.”
'Horses for Courses' in demand forecasting (2014, methodology): “ Running sliding simulations in the forecast evaluations (Tashman, 2000).”
Forecasting spot and forward prices in the international freight market (2007, methodology): “Their tests use overlapping forecast intervals, which Tashman (2000) argues may bias forecast evaluation, and we have been careful to design our forecast evaluation procedures to avoid this problem.”
To combine or not to combine: selecting among forecasts and their combinations (2005, methodology): “We use a simple model selection criterion based on the sliding simulation idea of Makridakis (1990), also described by Tashman (2000).”
Accurate On-line Support Vector Regression (2003, background): “A typical online time-series prediction scenario can be represented as follows (Tashman, 2000): (1) Given a time series { ( ), 1, 2,3 } x t t = and prediction origin O, construct a set of training samples, , O B A , from the segment of time series”
Intermittent demand forecasts with neural networks (2013, methodology): “Therefore 98 out-of-sample multi-step ahead forecasts are produced and their respective errors calculated using MAE, in a rolling origin scheme as discussed by Tashman (2000).”
EXPONENTIAL SMOOTHING: THE STATE OF THE ART, PART II (2006, neutral): “The question of whether out-of-sample criteria should be used for method selection is beyond our scope—see Tashman (2000) for a review.”
The use of analogies in forecasting the annual sales of new electronics products (2013, background): “All of this is a concern because, as Tashman (2000) has pointed out, extensive out-of-sample testing is needed to allow reliable inferences to be drawn about the accuracy of a given method.”