Distribution Free Prediction Sets

By Jing Lei, James M. Robins, Larry A. Wasserman. 2013

268 citations recorded by the source corpus.

This article introduces a new approach to prediction by bringing together two different nonparametric ideas: distribution-free inference and nonparametric smoothing. Specifically, we consider the problem of constructing nonparametric tolerance/prediction sets. We start from the general conformal prediction approach, and we use a kernel density estimator as a measure of agreement between a sample point and the underlying distribution. The resulting prediction set is shown to be closely related to plug-in density level sets with carefully chosen cutoff values. Under standard smoothness conditions, we get an asymptotic efficiency result that is near optimal for a wide range of function classes. But the coverage is guaranteed whether or not the smoothness conditions hold and regardless of the sample size. The performance of our method is investigated through simulation studies and illustrated in a real data example.

Subfields: Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Deep Weighted Averaging Classifiers (2018, neutral): “Conformal methods refer to a broad set of ideas which aim to provide theoretical guarantees on error rates in classification or regression [26, 45, 47, 54].”
Conformalized Quantile Regression (2019, methodology): “To generate the training data, we draw n = 2000 independent, univariate predictor samples X i from the uniform distribution on the interval [1 , 5] .”
Distribution‐free prediction bands for non‐parametric regression (2014, methodology): “Another important class of methods is quantile regression (Koenker and Hallock, 2001) where it is assumed, in a non-parametric form, that the τ th quantile of Y given X=x is a smooth function fτ .”
With Malice Towards None: Assessing Uncertainty via Equalized Coverage (2019, methodology): “Such a procedure can be formulated by refining tools from conformal inference, a general methodology for constructing prediction intervals [18–24].”
Least Ambiguous Set-Valued Classifiers With Bounded Error Levels (2016, background): “Lei et al. (2014, 2013), Lei & Wasserman (2014), and Lei (2014) studied the conformal approach from the point of view of statistical optimality in the unsupervised, regression and binary classiﬁcation cases, respectively.”
Mitigating Neural Network Overconfidence with Logit Normalization (2022, methodology): “Conformal prediction based method (Lei et al., 2013) outputs the empty set as prediction in case of too high “atypicality”.”