What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams

By Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, Peter Szolovits. 2020

1,893 citations recorded by the source corpus.

Open domain question answering (OpenQA) tasks have been recently attracting more and more attention from the natural language processing (NLP) community. In this work, we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams. It covers three languages: English, simplified Chinese, and traditional Chinese, and contains 12,723, 34,251, and 14,123 questions for the three languages, respectively. We implement both rule-based and popular neural methods by sequentially combining a document retriever and a machine comprehension model. Through experiments, we find that even the current best method can only achieve 36.7%, 42.0%, and 70.1% of test accuracy on the English, traditional Chinese, and simplified Chinese questions, respectively. We expect MedQA to present great challenges to existing OpenQA systems and hope that it can serve as a platform to promote much stronger OpenQA models from the NLP community in the future.

Subfields: Artificial Intelligence, Question Answering & Reading Comprehension, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Mohit Bansal
  3. Luke S. Zettlemoyer
  4. Jianfeng Gao
  5. Yejin Choi
  6. Yoshua Bengio
  7. Quoc V. Le
  8. Furu Wei

In Question Answering & Reading Comprehension

  1. Benjamin Van Durme
  2. Daniel Khashabi
  3. Li Dong
  4. Furu Wei
  5. Jason Wei
  6. Christopher D. Manning
  7. Percy Liang
  8. Jure Leskovec

In Medical & Health AI

  1. Christopher D. Manning
  2. Percy Liang
  3. Jure Leskovec
  4. Michihiro Yasunaga
  5. Antoine Bosselut
  6. Hongyu Ren
  7. Xikun Zhang

Citations in the corpus