Conformalized Quantile Regression

By Yaniv Romano, Evan Patterson, E.J. Candes. Neural Information Processing Systems, 2019

1,038 citations recorded by the source corpus.

Conformal prediction is a technique for constructing prediction intervals that attain valid coverage in finite samples, without making distributional assumptions. Despite this appeal, existing conformal methods can be unnecessarily conservative because they form intervals of constant or weakly varying length across the input space. In this paper we propose a new method that is fully adaptive to heteroscedasticity. It combines conformal prediction with classical quantile regression, inheriting the advantages of both. We establish a theoretical guarantee of valid coverage, supplemented by extensive experiments on popular regression datasets. We compare the efficiency of conformalized quantile regression to other conformal methods, showing that our method tends to produce shorter intervals.

Subfields: Artificial Intelligence, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/6f9dc6f8519e927d948a13aa7ae0df336f443eb9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Adaptive Conformal Inference Under Distribution Shift (2021, methodology): “Then, we could use the method of conformal quantile regression (CQR) [28], which examines the signed distance between y and ﬁtted upper and lower quantiles through the score S ( X t , y ) = max { ˆ q ( X t ; α/ 2) − y, y − ˆ q ( X t ; 1 − α/ 2) } .”
Few-shot Conformal Prediction with Auxiliary Tasks (2021, background): “…conformal prediction in the context of various validity conditions, such as conditional, risk-controlling, admissible, or equalized coverage (Chernozhukov et al., 2019; Cauchois et al., 2020; Kivaranovic et al., 2020; Romano et al., 2019; 2020; Bates et al., 2020; Fisch et al., 2021, inter alia ).”
Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction (2021, methodology): “For instance, in Izbicki et al. (2019), the authors consider using the estimated conditional density as the conformal score, and in Romano et al. (2019), the authors use the conformal score based on estimated quantile functions.”
Classification with Valid and Adaptive Coverage (2020, methodology): “The work of [2] addresses this problem by applying quantile regression [18] to hold-out scores f̂ .”
A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification (2021, methodology): “We will next show how to incorporate uncertainty into regression problems with a continuous output, following the algorithm in [7].”
Conformal prediction with conditional guarantees (2023, positive): “Finally, in this context we have which is a tractable function.”
Distribution-Free, Risk-Controlling Prediction Sets (2021, background): “See [14, 16] for further discussion of this point in that context.”
With Malice Towards None: Assessing Uncertainty via Equalized Coverage (2019, methodology): “Then we fit a neural network regression function μ̂ on the training set, where the network architecture, optimization, and hyper-parameters are similar to those described and implemented in [24].”
Conformal inference of counterfactuals and individual treatment effects (2020, methodology): “For instance, with estimates q̂αlo(x) and q̂αhi(x) of the αlo-th and αhi-th conditional quantiles of Y | X = x, the conformal quantile regression (CQR) introduced in Romano et al. [2019] as a variant of the standard conformal inference, would produce an interval estimate of the form”
Conformal Risk Control (2022, background): “…algorithm, mainly targeting deviations from exchangeability (Tibshirani et al., 2019; Gibbs & Candes, 2021; Barber et al., 2022; Fannjiang et al., 2022) and improved conditional coverage (Barber et al., 2020; Romano et al., 2019; Guan, 2020; Romano et al., 2020; Angelopoulos et al., 2021b).”
Knowing what You Know: valid and validated confidence sets in multiclass and multilabel prediction (2020, methodology): “CQC is similar to Romano et al. (2019)’s Conformalized Quantile Regression (CQR): we estimate a quantile function of the scores, which we use to construct valid conﬁdence sets after conformalization.”
Conformal prediction beyond exchangeability (2022, background): “Conformal prediction was pioneered by Vladimir Vovk and various collaborators in the early 2000s; the book by Vovk et al. [2005] details their advances and remains a critical resource.”
Knowing what you know: valid confidence sets in multiclass and multilabel prediction (2020, background): “[27] originate for regression problems—and build feature-adaptive quantile predictors, which output sets of labels, allowing us to guarantee valid marginal coverage (independent of the data generating distribution) while better approximating the conditional coverage (1).”
A comparison of some conformal quantile regression methods (2019, methodology): “In the following, we utilize two alternative black-box quantile regressors, implemented and trained as in [1].”
Uncertainty Sets for Image Classifiers using Conformal Prediction (2020, background): “We say that a set-valued predictor C : Distribution-free guarantees on conditional coverage are impossible (Vovk, 2012; Lei & Wasserman, 2014), but many algorithms try to satisfy it approximately (Romano et al., 2019; 2020; Cauchois et al., 2020).”