♫ MuSiQue: Multihop Questions via Single-hop Question Composition
By Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal. 2021
972 citations recorded by the source corpus.
Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts. Can we create a question answering (QA) dataset that, by construction, requires proper multihop reasoning? To this end, we introduce a bottom–up approach that systematically selects composable pairs of single-hop questions that are connected, that is, where one reasoning step critically relies on information from another. This bottom–up methodology lets us explore a vast space of questions and add stringent filters as well as other mechanisms targeting connected reasoning. It provides fine-grained control over the construction process and the properties of the resulting k-hop questions. We use this methodology to create MuSiQue-Ans, a new multihop QA dataset with 25K 2–4 hop questions. Relative to existing datasets, MuSiQue-Ans is more difficult overall (3× increase in human–machine gap), and harder to cheat via disconnected reasoning (e.g., a single-hop model has a 30-point drop in F1). We further add unanswerable contrast questions to produce a more stringent dataset, MuSiQue-Full. We hope our datasets will help the NLP community develop models that perform genuine multihop reasoning.1
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Prompting & Reasoning, Question Answering & Reading Comprehension
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Ashish Sabharwal
- Mohit Bansal
- Luke S. Zettlemoyer
- Tushar Khot
- Hannaneh Hajishirzi
- Noah A. Smith
- Peter Clark
- Yejin Choi
In Computer Vision and Pattern Recognition
- Ashish Sabharwal
- Tushar Khot
- Daniel Khashabi
- Kyle Richardson
- Peter Clark
- Hannaneh Hajishirzi
- Noah A. Smith
- Sewon Min
In Prompting & Reasoning
- Ashish Sabharwal
- Tushar Khot
- Peter Clark
- Jonathan Herzig
- Kyle Richardson
- Daniel Khashabi
- Mor Geva
- Avi Caciularu
In Question Answering & Reading Comprehension
- Ashish Sabharwal
- Peter Clark
- Tushar Khot
- Kyle Richardson
- Harsh Trivedi
- Daniel Khashabi
- Niranjan Balasubramanian
- Mor Geva
Citations in the corpus
- Measuring and Narrowing the Compositionality Gap in Language Models (2022, methodology): “Self-ask improves over chain of thought by smaller margins on 2WikiMultiHopQA and Musique but by a large 11% (absolute) on Bam-boogle.”
- Decomposed Prompting: A Modular Approach for Solving Complex Tasks (2022, methodology): “(4) On three multi-hop open-domain QA datasets (Yang et al., 2018; Ho et al., 2020; Trivedi et al., 2022), we can incorporate a symbolic retrieval (ElasticSearch) API as the handler for the retrieval sub-task leading to better results than CoT.”
- A Comprehensive Evaluation of Tool-Assisted Generation Strategies (2023, methodology): “, 2021) and MuSiQue (Trivedi et al., 2022b), which require reasoning about entity knowledge, and GSM8k (Cobbe et al.”
- Group-in-Group Policy Optimization for LLM Agent Training (2025, neutral): “5-1.5B-Instruct as the policy of the agent.”
- Knowledge Graph Prompting for Multi-Document Question Answering (2023, methodology): “2020) and MuSiQue (Trivedi et al. 2022b), and then prompt LLMs in four different strategies for the answer1.”
- RULER: What's the Real Context Size of Your Long-Context Language Models? (2024, background): “The majority of existing QA datasets (Rajpurkar et al., 2018; Yang et al., 2018; Trivedi et al., 2022) are designed to answer questions based on short passages.”
- LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding (2023, methodology): “The English test samples are built from three Wikipedia-based multi-hop QA datasets: HotpotQA (Yang et al., 2018), 2WikiMultihopQA (Ho et al., 2020), and MuSiQue (Trivedi et al., 2022).”
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions (2022, methodology): “, 2020), MuSiQue (Trivedi et al., 2022) and IIRC (Ferguson et al.”
- To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning (2024, neutral): “Because answer spans in MuSiQue can be paraphrased in many different ways, we use GPT-4o to judge if two answer spans are equivalent.”
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025, neutral): “(2) Multi-Hop Question Answering : HotpotQA (Yang et al., 2018), 2WikiMultiHopQA (Ho et al., 2020), Musique (Trivedi et al., 2022b), and Bamboogle (Press et al., 2022).”
- Gated Delta Networks: Improving Mamba2 with Delta Rule (2024, neutral): “Long context understanding We evaluate on 14 tasks from Longbench ( Bai et al., 2023), encompassing: narrative comprehension (Narrative QA (Koˇciský et al., 2018)), scientific understanding (QasperQA (Dasigi et al., 2021)), multi-hop reasoning (MultiField QA, HotpotQA (Yang et al., 2018), 2WikiMulti ”
- Learning to Solve Complex Tasks by Talking to Agents (2021, methodology): “This dataset consists of multi-hop questions where the reasoning needed to answer the question is E xplicitly described in the question itself (Yang et al., 2018; Ho et al., 2020; Trivedi et al., 2021).”
- A Matryoshka Hierarchical RAG for Efficient Multi-Hop Question Answering (2026, unknown): “We evaluated the quality of the answers using Exact Match (EM) and token-level F1, computed against the gold answers [40, 10, 36].”