On-line predictive linear regression

By Vladimir G Vovk, Ilia Nouretdinov, A. Gammerman. 2005

108 citations recorded by the source corpus.

We consider the on-line predictive version of the standard problem of linear regression; the goal is to predict each consecutive response given the corresponding explanatory variables and all the previous observations. The standard treatment of prediction in linear regression analysis has two drawbacks: (1) the classical prediction intervals guarantee that the probability of error is equal to the nominal significance level $\varepsilon$, but this property per se does not imply that the long-run frequency of error is close to $\varepsilon$; (2) it is not suitable for prediction of complex systems as it assumes that the number of observations exceeds the number of parameters. We state a general result showing that in the on-line protocol the frequency of error for the classical prediction intervals does equal the nominal significance level, up to statistical fluctuations. We also describe alternative regression models in which informative prediction intervals can be found before the number of observations exceeds the number of parameters. One of these models, which only assumes that the observations are independent and identically distributed, is popular in machine learning but greatly underused in the statistical theory of regression.

Subfields: Artificial Intelligence, Management Science and Operations Research, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/cedc5805829f21c5782b781e329472aaebd81ab8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Classification with Valid and Adaptive Coverage (2020, methodology): “3 Related work We build upon conformal inference [12, 24, 26] and take inspiration from [3, 5, 8–10, 13, 18] which made conformal prediction for regression problems adaptive to heteroscedasticity, thus bringing it closer to conditional coverage [20].”
A comparison of some conformal quantile regression methods (2019, background): “|X and building a fixed-width band around it (Vovk et al., 1999; Vovk et al., 2005; Vovk et al., 2009; Lei et al., 2018).”
Conformal Prediction Under Covariate Shift (2019, background): “The deﬁnitive reference is the book by Vovk et al. (2005); see also Shafer and Vovk (2008); Vovk et al. (2009); Vovk (2013); Burnaev and Vovk (2014), and http://www.alrw.net for an often-updated list of conformal prediction work by Vovk and colleagues.”
With Malice Towards None: Assessing Uncertainty via Equalized Coverage (2019, methodology): “Such a procedure can be formulated by refining tools from conformal inference, a general methodology for constructing prediction intervals [18–24].”
Distribution-Free Predictive Inference for Regression (2016, methodology): “The conformal prediction framework was originally proposed as a sequential approach for forming prediction intervals, by Vovk, Gammerman, and Shafer (2005) and Vovk, Nouretdinov, and Gammerman (2009).”
Distribution Free Prediction Sets (2013, methodology): “…prediction method (Vovk, Gammerman, and Shafer 2005; Shafer and Vovk 2008) is a general approach for constructing distribution-free, sequential prediction sets using exchangeability and is usually applied to sequential classification and regression problems (Vovk, Nouretdinov, and Gammerman 2009).”
Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction (2021, background): “…+1 , Y n +1 ) as the ( n + 1) th observation, CP achieves (1.1) under the assumption that Z n +1 is also independently generated from P XY , without additional distributional assumptions on P XY itself (Vovk et al. 2005, Shafer & Vovk 2008, Vovk et al. 2009, Lei & Wasserman 2014, Lei et al. 2018).”