TruthfulQA: Measuring How Models Mimic Human Falsehoods

By Stephanie C. Lin, Jacob Hilton, Owain Evans. 2021

3,631 citations recorded by the source corpus.

We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning using training objectives other than imitation of text from the web.

Subfields: Artificial Intelligence, Safety Research, Large Language Models, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Jianfeng Gao
  4. Percy Liang
  5. Yejin Choi
  6. Graham Neubig
  7. Quoc V. Le
  8. Yoshua Bengio

In Safety Research

  1. Umang Bhatt
  2. Ethan Perez
  3. Meg Tong
  4. Dani Yogatama
  5. Yuanshun Yao
  6. Xiaojun Xu
  7. Qi Liu
  8. Phil Blunsom

In Large Language Models

  1. Luke S. Zettlemoyer
  2. Yejin Choi
  3. Noah A. Smith
  4. Hannaneh Hajishirzi
  5. Graham Neubig
  6. Danqi Chen
  7. Mohit Bansal
  8. Hinrich Schütze

In Evaluation & Benchmarks

  1. Mohit Bansal
  2. Yejin Choi
  3. Samuel R. Bowman
  4. Hannaneh Hajishirzi
  5. Luke S. Zettlemoyer
  6. Noah A. Smith
  7. Douwe Kiela
  8. Graham Neubig

Citations in the corpus