BBQ: A hand-built bias benchmark for question answering

By Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, Sam Bowman, Samuel R. Bowman. 2021

808 citations recorded by the source corpus.

It is well documented that NLP models learn social biases, but little work has been done on how these biases manifest in model outputs for applied tasks like question answering (QA). We introduce the Bias Benchmark for QA (BBQ), a dataset of question-sets constructed by the authors that highlight attested social biases against people belonging to protected classes along nine social dimensions relevant for U.S. English-speaking contexts. Our task evaluate model responses at two levels: (i) given an under-informative context, we test how strongly responses reflect social biases, and (ii) given an adequately informative context, we test whether the model’s biases override a correct answer choice. We find that models often rely on stereotypes when the context is under-informative, meaning the model’s outputs consistently reproduce harmful biases in this setting. Though models are more accurate when the context provides an informative answer, they still rely on stereotypes and average up to 3.4 percentage points higher accuracy when the correct answer aligns with a social bias than when it conflicts, with this difference widening to over 5 points on examples targeting gender for most models tested.

Subfields: Artificial Intelligence, Evaluation & Benchmarks, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Evaluation & Benchmarks

In Safety, Robustness & Interpretability

Citations in the corpus

Mixtral of Experts (2024, methodology): “To identify possible flaws to be by fine-tuning / preference modeling, we measure the base model performance on Bias Benchmark for QA (BBQ) [24] and Bias in Open-Ended Language Generation Dataset (BOLD) [10].”
Emergent Abilities of Large Language Models (2022, neutral): “…which measures gender bias in occupations such as “nurse” or “electrician,” scaling has improved performance so far (Du et al., 2021; Chowdhery et al., 2022), though BIG-Bench (2022) found in BBQ bias benchmark (Parrish et al., 2022) that bias can increase with scaling for ambiguous contexts.”
Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, background): “Evaluations can also be specific to a certain function of a modality, such as question-answering in language [175].”
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (2023, methodology): “…methods This method encompasses a boarder scope, including assessments of morality, bias and values, which is also more prevalent in evaluations of LMs. Parrish et al. (2022) proposed BBQ, designed for evaluating models’ bias in question-answering, involving ambiguous and disambiguated contexts.”
The Capacity for Moral Self-Correction in Large Language Models (2023, methodology): “4 To measure stereotyping, we use two well-known stereotyping benchmarks, BBQ [40] (§3.”
Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “We opt to primarily report dialogue few-shot results, in which we concatenate the dialogue prompt with 𝐾 = 5 examples of a user asking a question from BBQ, and the model responding with the correct answer (results are generally similar with 𝐾 = 2 to 𝐾 = 5, so long as the examples include both amb”
PaLM 2 Technical Report (2023, methodology): “4% accuracy), but that 3% of all disambiguated questions produce a form of representational harm by reinforcing social bias (Parrish et al., 2021).”
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings (2022, background): “Current benchmarks often measure bias in speciﬁc downstream tasks (e.g. Machine Translation (Stanovsky et al., 2019), Question Answering (Parrish et al., 2021), or Text Generation (Dhamala et al., 2021)), while others focus on bias in LLMs more generally (e.g Kurita et al., 2019; Nadeem et al., 2020”
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models (2022, background): “Parrish et al. [83] propose two metrics to score answers in ambiguous and unambiguous contexts.”
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset (2023, background): “BBQ [40] examines social biases against various identity groups in QA tasks.”
Inverse Scaling: When Bigger Isn't Better (2023, background): “Srivastava et al. (2022) found that larger LMs showed more bias against particular genders, religions, races, ethnicities, and nationalities in BIG-Bench tasks involving ambiguous contexts such as the BBQ dataset (Parrish et al., 2022); see Figure 12 of Srivastava et al. (2022).”
When Not to Trust Language Models: Investigating Effectiveness and Limitations of Parametric and Non-Parametric Memories (2022, background): “On the other hand, its reliance on their parameters to encode a wealth of world knowledge learned has several limitations: they often struggle to memorize less popular knowledge and can result in biased predictions (Li et al., 2020; Parrish et al., 2022) and their knowledge gets obsolete quickly.”
OpenAI o1 System Card (2024, positive): “For example, we use a range of public and internally constructed evaluations to measure o1 on tasks such as propensity to generate disallowed content, performance on tasks relevant to demographic fairness [5, 6], tendency to hallucinate [7], and presence of dangerous capabilities [8, 9].”
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories (2022, neutral): “Parrish et al. (2022) established that models often rely on stereotypes to answer in uncertain cases, so our results indicate that LMs are likely to rely on stereotypes disproportionately for minority groups.”
A Comprehensive Overview of Large Language Models (2023, neutral): “SQuAD [336], BoolQ [337], SQUADv2 [338], DROP [339], RTE [340], WebQA [341], CMRC2017 [342], CMRC2018 [343], CMRC2019 [344], COTE-BD [345], COTE-DP [345], COTE-MFW [345], MultiRC [346], Natural Questions [347], CNSE [301], DRCD [348], DuReader [349], Dureader robust [350], DuReader-QG [349], SciQ [3”