TruthfulQA: Measuring How Models Mimic Human Falsehoods
By Stephanie C. Lin, Jacob Hilton, Owain Evans. 2021
3,631 citations recorded by the source corpus.
We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning using training objectives other than imitation of text from the web.
Subfields: Artificial Intelligence, Safety Research, Large Language Models, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Luke S. Zettlemoyer
- Jianfeng Gao
- Percy Liang
- Yejin Choi
- Graham Neubig
- Quoc V. Le
- Yoshua Bengio
In Safety Research
In Large Language Models
- Luke S. Zettlemoyer
- Yejin Choi
- Noah A. Smith
- Hannaneh Hajishirzi
- Graham Neubig
- Danqi Chen
- Mohit Bansal
- Hinrich Schütze
In Evaluation & Benchmarks
- Mohit Bansal
- Yejin Choi
- Samuel R. Bowman
- Hannaneh Hajishirzi
- Luke S. Zettlemoyer
- Noah A. Smith
- Douwe Kiela
- Graham Neubig
Citations in the corpus
- Training Compute-Optimal Large Language Models (2022, positive): “In stark contrast with the findings of [31], the large improvements (14.”
- LLaMA: Open and Efficient Foundation Language Models (2023, background): “Lin et al. (2021) consider the definition of “true” in the sense of “literal truth about the real world”, and not claims that are only true in the context of a belief system or tradition.”
- Llama 2: Open Foundation and Fine-Tuned Chat Models (2023, methodology): “We employ TruthfulQA (Lin et al., 2021) to measure how well our LLMs can generate reliable outputs that agree with factuality and common sense.”
- Emergent Abilities of Large Language Models (2022, neutral): “Overall, emergent abilities should probably be viewed as a function of many correlated variables.”
- GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022, methodology): “We also note that these benefits are not hypothetical, as a number of papers about the limits and ethics of LLMs has been explicitly enabled by the public release of previous models (Zhang et al., 2021; Kandpal et al., 2022; Carlini et al., 2022; Birhane et al., 2021; nostalgebraist, 2020; Meng et a”
- Pretraining Language Models with Human Preferences (2023, background): “, falsehoods (Lin et al., 2022), offensive comments (Gehman et al.”
- Large Language Models Are Human-Level Prompt Engineers (2022, methodology): “We apply our method on TruthfulQA (Lin et al., 2022) to see how APE-generated instructions can steer an LLM to generate answers with different styles, and study the trade-off between truthfulness and informativeness.”
- Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “, 2020), but such utterances may be factually incorrect or misleading (Maynez et al., 2020; Xu, 2020; Lin et al., 2022b; Bickmore et al., 2018; Daws, 2020).”
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned (2022, background): “…social biases (e.g., [47, 28, 1, 33, 7]), generating offensive or toxic outputs [25], leaking personally identifiable information from the training data [13], aiding in disinformation campaigns [12], generating extremist texts [37], spreading falsehoods [35], and more [9, 10, 18, 57, 22, 51].”
- WebGPT: Browser-assisted question-answering with human feedback (2021, methodology): “Third, we evaluate our model on TruthfulQA [Lin et al., 2021], an adversarial dataset of short-form questions.”
- A General Language Assistant as a Laboratory for Alignment (2021, methodology): “• The prompt reduces toxicity [GGS + 20] (figure 8) and seemingly leads larger models to be more accurate than smaller models on TruthfulQA [LHE21] (figure 6).”
- Code Llama: Open Foundation Models for Code (2023, methodology): “The TruthfulQA benchmark comprises 817 questions spread across 38 categories, encompassing topics such as health, finance, law, and politics (Lin et al., 2022).”
- Judging LLM-as-a-judge with MT-Bench and Chatbot Arena (2023, methodology): “Here, we evaluate several variants of models derived from LLaMA on MMLU [16], Truthful QA [22] (MC1), and MT-bench (GPT-4 judge).”
- Measuring Progress on Scalable Oversight for Large Language Models (2022, background): “However, they have also memorized large swaths of inaccurate, dated, or debunked research, as well as uninformed social media writing on medicine (see Lin et al., 2022).”
- Improving alignment of dialogue agents via targeted human judgements (2022, methodology): “TruthfulQA samples from GopherCite This subsection presents Sparrow RL@8 responses for the TruthfulQA Evans et al. (2021) questions shown as failure examples of GopherCite Rae et al. (2021).”
- Training Language Models with Language Feedback (2022, background): “A key problem with LMs is that they generate text that violates human preferences, such as LM-generated misinformation (Lin et al., 2021), offensive language (Gehman et al., 2020), and fac-tually incorrect outputs such as summaries (Stien-non et al., 2020 highly according to human preferences, or a…”