Chi-Squared Tests of Interval and Density Forecasts, and the Bank of England's Fan Charts

By K. Wallis. Social Science Research Network, 2003

201 citations recorded by the source corpus.

This paper reviews recently proposed likelihood ratio tests of goodness-of-fit and independence of interval forecasts. It recasts them in the framework of Pearson chi-squared statistics, and extends them to density forecasts. Two further recent developments are also incorporated, namely a more informative decomposition of the goodness-of-fit statistic, and the calculation of exact P-values. Examples considered are the US Survey of Professional Forecasters density forecasts of inflation and the Bank of England fan charts. This first evaluation of the Bank forecasts finds that the fan charts fan out too quickly, and the excessive concern with the upside risks was not justified.

Subfields: Economics and Econometrics, Finance

View Paper: https://www.semanticscholar.org/paper/ca917a0e8576b2ad601ed75e53412cf67e7040b3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Economics and Econometrics

In Finance

Citations in the corpus

Nowcasting GDP in Real Time: A Density Combination Approach (2014, unknown): “Other tests of uniformity employed are the Anderson–Darling (AD) test (see Noceti, Smith, and Hodges 2003) and a Pearson chi-squared test, as suggested by Wallis (2003).”
Combining forecast densities from VARs with uncertain instabilities (2010, methodology): “With this issue in mind, we follow Wallis (2003) and employ a Pearson chi-squared test which divides the range of the zτ,h into eight equiprobable classes and tests whether the resulting histogram is uniform.”
Wind power forecasting : state-of-the-art 2009. (2009, unknown): “There are several frameworks for interval forecast evaluation in the econometric forecasting community [292],[293] that are based on testing the hypothesis of correct conditional coverage of prediction intervals.”
Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules (2011, methodology): “…issued probabilistic forecasts of inflation rates and gross domestic product every quarter since February 1996 and November 1997, respectively, using fan charts to visualize the deciles of the predictive distributions (Wallis 2003, 2004; Clements 2004; Elder et al. 2005; Mitchell and Hall 2005).”
EVALUATING T HE BANK OF ENGLAND D ENSITY FORECASTS OF INFLATION (2004, unknown): “…own assessment of these forecasts has tended to focus on the closeness of some measure of the central tendency and the outcomes, 1 although Wallis (2002) shows how the Christoffersen (1998) likelihood ratio goodness-of-fit statistics for evaluating interval forecasts can be interpreted as…”
25 years of time series forecasting (2006, methodology): “Other approaches to interval and density evaluation are given by Wallis (2003) who proposed chi-squared tests for both intervals and densities, and Clements & Smith (2002) who discussed some simple but powerful tests when evaluating multivariate forecast densities.”
MODELLING ASYMMETRIC EXCHANGE RATE DEPENDENCE a (2006, methodology): “Christoffersen (1998) and Wallis (2003) modeled πjt as a first-order Markov chain, whereas Engle and Manganelli (2004) used a linear probability model.”
Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination (2021, methodology): “Wallis (2003) and Wilks (2019, p. 769) suggest chi-square tests for this purpose, which depend on binning, and thus are subject to the aforementioned instabilities.”
Pooling of Forecasts (2004, unknown): “Wallis, K. F. (2003).”