♫ MuSiQue: Multihop Questions via Single-hop Question Composition

By Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal. 2021

972 citations recorded by the source corpus.

Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts. Can we create a question answering (QA) dataset that, by construction, requires proper multihop reasoning? To this end, we introduce a bottom–up approach that systematically selects composable pairs of single-hop questions that are connected, that is, where one reasoning step critically relies on information from another. This bottom–up methodology lets us explore a vast space of questions and add stringent filters as well as other mechanisms targeting connected reasoning. It provides fine-grained control over the construction process and the properties of the resulting k-hop questions. We use this methodology to create MuSiQue-Ans, a new multihop QA dataset with 25K 2–4 hop questions. Relative to existing datasets, MuSiQue-Ans is more difficult overall (3× increase in human–machine gap), and harder to cheat via disconnected reasoning (e.g., a single-hop model has a 30-point drop in F1). We further add unanswerable contrast questions to produce a more stringent dataset, MuSiQue-Full. We hope our datasets will help the NLP community develop models that perform genuine multihop reasoning.1

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Ashish Sabharwal
  2. Mohit Bansal
  3. Luke S. Zettlemoyer
  4. Tushar Khot
  5. Hannaneh Hajishirzi
  6. Noah A. Smith
  7. Peter Clark
  8. Yejin Choi

In Computer Vision and Pattern Recognition

  1. Ashish Sabharwal
  2. Tushar Khot
  3. Daniel Khashabi
  4. Kyle Richardson
  5. Peter Clark
  6. Hannaneh Hajishirzi
  7. Noah A. Smith
  8. Sewon Min

In Prompting & Reasoning

  1. Ashish Sabharwal
  2. Tushar Khot
  3. Peter Clark
  4. Jonathan Herzig
  5. Kyle Richardson
  6. Daniel Khashabi
  7. Mor Geva
  8. Avi Caciularu

In Question Answering & Reading Comprehension

  1. Ashish Sabharwal
  2. Peter Clark
  3. Tushar Khot
  4. Kyle Richardson
  5. Harsh Trivedi
  6. Daniel Khashabi
  7. Niranjan Balasubramanian
  8. Mor Geva

Citations in the corpus