In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?

By A. Inoue, Lutz Kilian. Social Science Research Network, 2005

714 citations recorded by the source corpus.

Abstract It is widely known that significant in-sample evidence of predictability does not guarantee significant out-of-sample predictability. This is often interpreted as an indication that in-sample evidence is likely to be spurious and should be discounted. In this paper, we question this interpretation. Our analysis shows that neither data mining nor dynamic misspecification of the model under the null nor unmodelled structural change under the null are plausible explanations of the observed tendency of in-sample tests to reject the no-predictability null more often than out-of-sample tests. We provide an alternative explanation based on the higher power of in-sample tests of predictability in many situations. We conclude that results of in-sample tests of predictability will typically be more credible than results of out-of-sample tests.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/6036cca7101342f5e0bebfc268e5f7bd9fd61af3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

OIL PRICE SHOCKS AND INDUSTRIAL PRODUCTION: IS THE RELATIONSHIP LINEAR? (2011, background): “[25] Inoue, Atsushi and Lutz Kilian (2004) “In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?”, Econometric Reviews, 23(4), 371-402.”
Time series momentum: Is it there? (2020, background): “Although no strict relation exists between the in-sample and out-of-sample performance (Inoue and Kilian, 2005), the last column of Table 2 and Panel B of Fig.”
Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average? (2008, background): “While it is not clear how much weight should be placed on out-of-sample statistics in judging the predictability of stock returns [ Inoue and Kilian (2004) ], in this paper we take up Goyal and Welch’s challenge and ask whether standard variables could have been used in real time to forecast 20th an”
Macro Factors in Bond Risk Premia (2005, methodology): “14Some researchers have used federal funds futures rates to measure the nancial markets expectation of future federal funds rates changes (Krueger and Kuttner (1996), Rudebusch (1998), Brunner (2000), Kuttner (2001), Bernanke and Kuttner (2003)).”
Predictability of Returns and Cash Flows (2011, neutral): “Inoue and Kilian (2004), Campbell and Thompson (2008), and Kelly and Pruitt (2010) have a different take on the out-of-sample tests and evidence.”
A Test for Superior Predictive Ability (2005, methodology): “The implementation is based on the stationary bootstrap of Politis and Romano (1994), but it is straightforward to modify the implementation to the block bootstrap of Künsch (1989). Although there are arguments that favor the block bootstrap over the stationary bootstrap (see Lahiri 1999), these adv”
Do High-Frequency Financial Data Help Forecast Oil Prices? The Midas Touch at Work (2015, background): “This point was first made in Inoue and Kilian (2004) and has become widely accepted in recent years.”
Manager sentiment and stock returns (2019, methodology): “Following Kandel and Stambaugh (1996), Campbell and Thompson (2008) and Ferreira and Santa-Clara (2011), among others, we compute the certainty equivalent return (CER) gain and Sharpe Ratio for a meanvariance investor who optimally allocates across equities and the risk-free asset using the outof-sa”
Reality Checks and Comparisons of Nested Predictive Models (2012, background): “Our paper extends Inoue and Kilian (2004) in three dimensions.”