Selective Question Answering under Domain Shift

By Amita Kamath, Robin Jia, Percy Liang. 2020

272 citations recorded by the source corpus.

To avoid giving wrong answers, question answering (QA) models need to know when to abstain from answering. Moreover, users often ask questions that diverge from the model’s training data, making errors more likely and thus abstention more critical. In this work, we propose the setting of selective question answering under domain shift, in which a QA model is tested on a mixture of in-domain and out-of-domain data, and must answer (i.e., not abstain on) as many questions as possible while maintaining high accuracy. Abstention policies based solely on the model’s softmax probabilities fare poorly, since models are overconfident on out-of-domain inputs. Instead, we train a calibrator to identify inputs on which the QA model errs, and abstain when it predicts an error is likely. Crucially, the calibrator benefits from observing the model’s behavior on out-of-domain data, even if from a different domain than the test data. We combine this method with a SQuAD-trained QA model and evaluate on mixtures of SQuAD and five other QA datasets. Our method answers 56% of questions while maintaining 80% accuracy; in contrast, directly using the model’s probabilities only answers 48% at 80% accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Break-It-Fix-It: Unsupervised Learning for Program Repair (2021, background): “Such domain shifts typically occur across related but different datasets (Torralba & Efros, 2011; Fang et al., 2013; Venkateswara et al., 2017; Yu et al., 2018b; 2019; Peng et al., 2019; Kamath et al., 2020), as well as from synthetic data to real data (including sim2real) (Wang et al.”
Improving alignment of dialogue agents via targeted human judgements (2022, background): “This ability can be viewed as a selective prediction of using evidence (or prediction with a reject option) (El-Yaniv and Wiener, 2010; Geifman and El-Yaniv, 2017, 2019; Kamath et al., 2020).”
Teaching language models to support answers with verified quotes (2022, background): “This setup could be described as “selective prediction” (also known as prediction with a reject option) (El-Yaniv et al., 2010; Geifman and El-Yaniv, 2017, 2019; Kamath et al., 2020).”
WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020, positive): “Indeed, prior work has shown that selective prediction can improve model accuracy under distribution shifts (Pimentel et al., 2014; Hendrycks and Gimpel, 2017; Liang et al., 2018; Ovadia et al., 2019; Feng et al., 2019; Kamath et al., 2020).”
Linguistic calibration through metacognition: aligning dialogue agent responses with expected correctness (2020, background): “The work of Kamath et al. (2020) attempts to correct overconﬁdence in neural network models, by training QA models to abstain from answering questions in which they are likely to err, using probabilistic calibration (see next paragraph).”
Selective Classification Can Magnify Disparities Across Groups (2020, background): “Prior work has shown that, across a broad array of applications, more confident predictions tend to be more accurate (Hanczar & Dougherty, 2008; Yu et al., 2011; Toplak et al., 2014; Mozannar & Sontag, 2020; Kamath et al., 2020).”
Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings (2022, methodology): “Calibration: For calibrating QA models, we use input length, predicted answer length, and softmax probabilities of top 5 predictions as the features (similar to Kamath et al. (2020)).”
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning (2022, methodology): “These confidence scores can be used in a “selective QA” setting (Kamath et al., 2020), where the model can abstain on a certain fraction of questions where it assigns low confidence to its answers.”
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model (2022, methodology): “To test this idea, we perform an evaluation of selective prediction , in which answers are produced only when they are likely to be correct (Ro-driguez et al., 2019; Kamath et al., 2020).”
Teaching Models to Express Their Uncertainty in Words (2022, background): “Kamath et al. (2020) train an auxiliary “calibrator” to predict whether the primary model correctly answers any given question using a mix of in-domain and out-of-domain data.”
Uncertainty Estimation for Language Reward Models (2022, background): “Moreover, language models’ uncertainty is unreliable out-of-domain [10, 21], and is poorly calibrated even in-domain for some tasks such as Q&A [20].”
Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models (2023, background): “For instance, Kamath et al. (2020) examines the selective question-answering task as a multiple-choice problem, reducing it to a de facto classification task rather than directly engaging with free-form generation.”
Can NLI Models Verify QA Systems' Predictions? (2021, methodology): “For example, in the selective QA setting (Kamath et al., 2020), our approach improves the F1 score of the base QA model from 81.6 to 87.1 when giving answers on the 20% of questions it is most conﬁdent about.”
The Unreliability of Explanations in Few-Shot In-Context Learning (2022, methodology): “These conﬁdence scores can be used in a “selective QA” setting (Kamath et al., 2020), where the model can abstain on a certain fraction of questions where it assigns low conﬁdence to its answers.”
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering (2020, methodology): “…as part-of-speech tagging and named entity recognition (Jagannatha and Yu, 2020), natural language understanding tasks such as natural language inference, paraphrase detection, extractive question answering, and text clas-siﬁcation (Desai and Durrett, 2020; Kamath et al., 2020; Kong et al., 2020).”