MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

By Daniel Fernando Campos, Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, Li Deng, Bhaskar Mitra. 2016

3,491 citations recorded by the source corpus.

This paper presents our recent work on the design and development of a new, large scale dataset, which we name MS MARCO, for MAchine Reading COmprehension. This new dataset is aimed to overcome a number of well-known weaknesses of previous publicly available datasets for the same task of reading comprehension and question answering. In MS MARCO, all questions are sampled from real anonymized user queries. The context passages, from which answers in the dataset are derived, are extracted from real web documents using the most advanced version of the Bing search engine. The answers to the queries are human generated. Finally, a subset of these queries has multiple answers. We aim to release one million queries and the corresponding answers in the dataset, which, to the best of our knowledge, is the most comprehensive real-world dataset of its kind in both quantity and quality. We are currently releasing 100,000 queries with their corresponding answers to inspire work in reading comprehension and question answering along with gathering feedback from the research community.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Luke S. Zettlemoyer
  3. Jianfeng Gao
  4. Percy Liang
  5. Hannaneh Hajishirzi
  6. Graham Neubig
  7. Yejin Choi
  8. Kyunghyun Cho

In Information Systems

  1. Jimmy J. Lin
  2. Rodrigo Nogueira
  3. Jamie Callan
  4. Sebastian Hofstätter
  5. Zeynep Akkalyoncu Yilmaz
  6. Zhuyun Dai
  7. Allan Hanbury
  8. Andrew Yates

In Information Retrieval & Search

  1. Jimmy J. Lin
  2. Jamie Callan
  3. Wen-tau Yih
  4. Sebastian Hofstätter
  5. Luyu Gao
  6. Sebastian Riedel
  7. Hannaneh Hajishirzi
  8. Rodrigo Nogueira

In Evaluation & Benchmarks

  1. Mohit Bansal
  2. Hannaneh Hajishirzi
  3. Douwe Kiela
  4. Luke S. Zettlemoyer
  5. Matt Gardner
  6. Sebastian Riedel
  7. Robin Jia
  8. Jonathan Berant

Citations in the corpus