CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

By Alon Talmor, Jonathan Herzig, Nicholas Lourie, Jonathan Berant. 2019

2,559 citations recorded by the source corpus.

When answering a question, people often draw upon their rich world knowledge in addition to the particular context. Recent work has focused primarily on answering questions given some relevant document or context, and required very little general background. To investigate question answering with prior knowledge, we present CommonsenseQA: a challenging new dataset for commonsense question answering. To capture common sense beyond associations, we extract from ConceptNet (Speer et al., 2017) multiple target concepts that have the same semantic relation to a single source concept. Crowd-workers are asked to author multiple-choice questions that mention the source concept and discriminate in turn between each of the target concepts. This encourages workers to create questions with complex semantics that often require prior knowledge. We create 12,247 questions through this procedure and demonstrate the difficulty of our task with a large number of strong baselines. Our best baseline is based on BERT-large (Devlin et al., 2018) and obtains 56% accuracy, well below human performance, which is 89%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Jianfeng Gao
  4. Yejin Choi
  5. Percy Liang
  6. Noah A. Smith
  7. Graham Neubig
  8. Quoc V. Le

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Xiaogang Wang
  5. Mohit Bansal
  6. Jitendra Malik
  7. Ming-Hsuan Yang
  8. Andrew Zisserman

In Natural Language Processing

  1. Mohit Bansal
  2. Yejin Choi
  3. Luke S. Zettlemoyer
  4. Graham Neubig
  5. Jianfeng Gao
  6. Hannaneh Hajishirzi
  7. Percy Liang
  8. Noah A. Smith

In Question Answering & Reading Comprehension

  1. Ashish Sabharwal
  2. Tushar Khot
  3. Peter Clark
  4. Hannaneh Hajishirzi
  5. Kyle Richardson
  6. Jonathan Berant
  7. Yejin Choi
  8. Xiang Ren

Citations in the corpus