Look at the First Sentence: Position Bias in Question Answering

By Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim, Jaewoo Kang. 2020

123 citations recorded by the source corpus.

Many extractive question answering models are trained to predict start and end positions of answers. The choice of predicting answers as positions is mainly due to its simplicity and effectiveness. In this study, we hypothesize that when the distribution of the answer positions is highly skewed in the training set (e.g., answers lie only in the k-th sentence of each passage), QA models predicting answers as positions learn spurious positional cues and fail to give answers in different positions. We first illustrate this position bias in popular extractive QA models such as BiDAF and BERT and thoroughly examine how position bias propagates through each layer of BERT. To safely deliver position information without position bias, we train models with various de-biasing methods including entropy regularization and bias ensembling. Among them, we found that using the prior distribution of answer positions as a bias model is very effective at reducing position bias recovering the performance of BERT from 35.24% to 81.17% when trained on a biased SQuAD dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Judging LLM-as-a-judge with MT-Bench and Chatbot Arena (2023, background): “This bias is not unique to our context and has been seen in human decision-making [3, 29] and other ML domains [18, 35].”
Analyzing Dynamic Adversarial Training Data in the Limit (2021, methodology): “Traditional crowdsourcing methods often yield datasets that lack diversity, contain spurious correlations, and are easy for existing models (Gururangan et al., 2018; Poliak et al., 2018; Geva et al., 2019; Ko et al., 2020; Potts et al., 2021).”
What Do Models Learn from Question Answering Datasets? (2020, background): “…answering has found that state-of-the-art models can get good performance on incomplete input (Agrawal et al., 2016; Sugawara et al., 2018; Niven and Kao, 2019), under-rely on important words, (Mudrakarta et al., 2018), and over-rely on simple heuristics (Weissenborn et al., 2017; Ko et al., 2020).”
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks (2022, background): “, a model’s positional bias (Murray and Chiang, 2018; Ko et al., 2020) (bias to resolve “she" to a closeby entity) or spurious correlations (Schlegel et al.”
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning (2023, methodology): “To further address the positional bias (Ko et al. 2020; Wang et al. 2023a), we send the responses of two models to the judge twice with different orders and compare their scores.”
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine (2023, background): “While less severe than other foundation models, GPT-4 can exhibit a propensity to favor certain options in multiple choice answers over others (regardless of the option content), i.e., the model can show position bias [1, 16, 40].”
JudgeLM: Fine-tuned Large Language Models are Scalable Judges (2023, unknown): “Position bias means that the LLM judges prefer answers in a certain position and it widely exists in natural language processing tasks (Ko et al., 2020; Wang et al., 2018) and decision-making of humans (Blunch, 1984; Raghubir & Valenzuela, 2006).”