MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering

By Ankit Pal, Logesh Kumar Umapathi, Malaikannan Sankarasubbu. 2022

777 citations recorded by the source corpus.

This paper introduces MedMCQA, a new large-scale, Multiple-Choice Question Answering (MCQA) dataset designed to address real-world medical entrance exam questions. More than 194k high-quality AIIMS \&NEET PG entrance exam MCQs covering 2.4k healthcare topics and 21 medical subjects are collected with an average token length of 12.77 and high topical diversity. Each sample contains a question, correct answer(s), and other options which requires a deeper language understanding as it tests the 10+ reasoning abilities of a model across a wide range of medical subjects \&topics. A detailed explanation of the solution, along with the above information, is provided in this study.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Galactica: A Large Language Model for Science (2022, neutral): “(Jin et al., 2019; Pal et al., 2022).”
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics (2023, neutral): “For instance, Med-PaLM 2 [1] approached or exceeded state-of-the-art performance across MedMCQA [38], PubMedQA [39], and MMLU [40] clinical topics QA datasets.”
Capabilities of GPT-4 on Medical Challenge Problems (2023, methodology): “…question, where A is yes, B is no, and C is maybe, matching the testing method done by Li´evin et al. [LHW22] • MedMCQA : MedMCQA is a dataset of multiple choice questions sourced from mock exams and published past exams of two Indian medical school entrance exams called AIIMS and NEET-PG [PUS22].”
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2022, methodology): “The other 30 embedding training datasets come from the Sentence Transformers embedding data, 2 KILT (Petroni et al., 2021), and MedMCQA (Pal et al., 2022).”
ExpertQA: Expert-Curated Questions and Attributed Answers (2023, background): “Specifically, existing work has presented datasets for the medical domain (Tsatsaronis et al., 2015; Pampari et al., 2018; Jin et al., 2019, 2021; Pal et al., 2022), legal (Guha et al.”
Toward expert-level medical question answering with large language models (2025, positive): “A majority of these approaches involve smaller language models trained using domain-specific data (BioLinkBert 10 , DRAGON 11 , PubMedGPT 12 , PubMedBERT 13 , BioGPT 14 ), resulting in steady improvements in performance on benchmark datasets such as MedQA (United States Medical Licensing Examination”
Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health (2023, background): “A wide variety of biomedical QA datasets have been introduced over the past decade, including BioASQ [60, 61], MedMCQA [44], MedQA (USMLE) [43], PubMedQA [62], GeneTuring [63].”
MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning (2023, neutral): “Examples illustrating interpretability are shown in Appendix B. (iii) Experimental results on nine datasets”
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine (2023, background): “• MedMCQA [25] presents mock and historic exam questions in the style of two Indian medical school entrance exams—the AIIMS and NEET-PG.”
Baichuan 2: Open Large-scale Language Models (2023, background): “, 2021) and MedMCQA (Pal et al., 2022), as well as average scores from medical-related disciplines in C-Eval (val), MMLU, and CMMLU (abbreviated as CMC).”
LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models (2023, background): “The evaluations on three medical datasets revealed significant enhancements in both in-domain (PubMedQA [20], MedMCQA [25]) and out-of-domain (MedQA-USMLE [19]) datasets.”
Can large language models reason about medical questions? (2022, background): “…question answering datasets: USMLE, whichincludesdifﬁcult real-world medicalques-tions targeting medical professionals 32 ; the MedMCQA, which gathers questions from medical school entrance exams 44 ; and the PubMedQA, which includes reading comprehension questions about PubMed abstracts.”
The Claude 3 Model Family: Opus, Sonnet, Haiku (methodology): “We use PubmedQA [23], BioASQ [69], USMLE [70], and MedMCQA [71].”