Conformal prediction beyond exchangeability
By Rina Foygel Barber, E.J. Candes, Aaditya Ramdas, Robert Tibshirani. Annals of Statistics, 2022
494 citations recorded by the source corpus.
Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. However, exchangeability is often violated when predictive models are deployed in practice. For example, if the data distribution drifts over time, then the data points are no longer exchangeable; moreover, in such settings, we might want to use a nonsymmetric algorithm that treats recent observations as more relevant. This paper generalizes conformal prediction to deal with both aspects: we employ weighted quantiles to introduce robustness against distribution drift, and design a new randomization technique to allow for algorithms that do not treat data points symmetrically. Our new methods are provably robust, with substantially less loss of coverage when exchangeability is violated due to distribution drift or other challenging features of real data, while also achieving the same coverage guarantees as existing conformal prediction methods if the data points are in fact exchangeable. We demonstrate the practical utility of these new tools with simulations and real-data experiments on electricity and election forecasting.
Subfields: Artificial Intelligence, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/ba98e89eb2afea1155c65d2ae26903d593c2fe22
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Statistics and Probability
Citations in the corpus
- Conformal prediction with conditional guarantees (2023, unknown): “In particular, we show how our method can be used to satisfy two popular coverage desiderata: group-conditional coverage and coverage under covariate shift.”
- A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification (2021, background): “A weighted version of conformal prediction pioneered by [26] provides tools for addressing non-exchangeable data, most notably slowly changing time-series.”
- Conformal Risk Control (2022, unknown): “A researcher may want to control E [ L i ( λ ; γ )] at level α ( γ ) .”