Combining Retrieval, Statistics, and Inference to Answer Elementary Science Questions

By Peter Clark, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter D. Turney, Daniel Khashabi. 2016

148 citations recorded by the source corpus.

What capabilities are required for an AI system to pass standard 4th Grade Science Tests? Previous work has examined the use of Markov Logic Networks (MLNs) to represent the requisite background knowledge and interpret test questions, but did not improve upon an information retrieval (IR) baseline. In this paper, we describe an alternative approach that operates at three levels of representation and reasoning: information retrieval, corpus statistics, and simple inference over a semi-automatically constructed knowledge base, to achieve substantially improved results. We evaluate the methods on six years of unseen, unedited exam questions from the NY Regents Science Exam (using only non-diagram, multiple choice questions), and show that our overall system’s score is 71.3%, an improvement of 23.8% (absolute) over the MLN-based method described in previous work. We conclude with a detailed analysis, illustrating the complementary strengths of each method in the ensemble. Our datasets are being released to enable further research.

Subfields: Artificial Intelligence, Retrieval-Augmented Generation, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Hannaneh Hajishirzi
  3. Luke S. Zettlemoyer
  4. Ashish Sabharwal
  5. Yejin Choi
  6. Percy Liang
  7. Jianfeng Gao
  8. Peter Clark

In Retrieval-Augmented Generation

  1. Peter Clark
  2. Ashish Sabharwal
  3. Tushar Khot
  4. Daniel Khashabi
  5. Sebastian Riedel
  6. Luke S. Zettlemoyer
  7. Ming Zhou
  8. Christopher D. Manning

In Natural Language Processing

  1. Ashish Sabharwal
  2. Peter Clark
  3. Tushar Khot
  4. Mohit Bansal
  5. Hannaneh Hajishirzi
  6. Matt Gardner
  7. Jonathan Berant
  8. Luke S. Zettlemoyer

Citations in the corpus