Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models

By Tushar Khot, Daniel Khashabi, Kyle Richardson, Peter Clark, Ashish Sabharwal. 2021

97 citations recorded by the source corpus.

We propose a general framework called Text Modular Networks(TMNs) for building interpretable systems that learn to solve complex tasks by decomposing them into simpler ones solvable by existing models. To ensure solvability of simpler tasks, TMNs learn the textual input-output behavior (i.e., language) of existing models through their datasets. This differs from prior decomposition-based approaches which, besides being designed specifically for each complex task, produce decompositions independent of existing sub-models. Specifically, we focus on Question Answering (QA) and show how to train a next-question generator to sequentially produce sub-questions targeting appropriate sub-models, without additional human annotation. These sub-questions and answers provide a faithful natural language explanation of the model’s reasoning. We use this framework to build ModularQA, a system that can answer multi-hop reasoning questions by decomposing them into sub-questions answerable by a neural factoid single-span QA model and a symbolic calculator. Our experiments show that ModularQA is more versatile than existing explainable systems for DROP and HotpotQA datasets, is more robust than state-of-the-art blackbox (uninterpretable) systems, and generates more understandable and trustworthy explanations compared to prior work.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Reasoning Like Program Executors (2022, background): “(Andreas et al., 2016), reading comprehension (Gupta et al., 2019; Khot et al., 2021), knowledge base question answering (Ren et al., 2021) and 3D rendering (Tian et al., 2019).”
Flexible Generation of Natural Language Deductions (2021, methodology): “Khot et al. (2021) use generative models to decompose a complex QA problem into a series of elementary steps that can be delegated to simpler models; this idea parallels the notion of decomposing reasoning into simple steps to be performed by generative operation models.”
CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, background): “…(4) learning to accumulate knowledge and avoid catas- trophic forgetting in an continual learning setup (Jin et al., 2021); (5) decomposing complex tasks into atomic tasks and exploring cross-task general- ization through the lens of compositionality (An- dreas et al., 2016; Khot et al., 2021).”
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2021, methodology): “The approach of text modular networks promises more interpretable answers to multi-hop questions (Khot et al., 2020; Jiang and Bansal, 2019; Gupta et al., 2020).”
Decomposed Prompting: A Modular Approach for Solving Complex Tasks (2022, methodology): “…for Multi-Step Reasoning Our work follows a long literature in NLP on neural modular modeling architectures (Andreas et al., 2016; Talmor & Berant, 2018; Jiang & Bansal, 2019; Gupta et al., 2020; Perez et al., 2020; Khot et al., 2021; Levine et al., 2022) for question-answering and other tasks.”
Learning to Solve Complex Tasks by Talking to Agents (2021, methodology): “Additionally these datasets contain sub-problems that may not be solvable by existing models, further disincentivis-ing development of compositional models (Khot et al., 2021).”
♫ MuSiQue: Multihop Questions via Single-hop Question Composition (2021, methodology): “Similar to prior decompositional approaches (Talmor and Berant, 2018; Min et al., 2019b; Qi et al., 2020; Khot et al., 2021), this model performs explicit, step-by-step multihop reasoning, by first predicting a decomposition of the input question Q into a DAG GQ containing single-hop questions, and ”
HELP ME THINK: A Simple Prompting Strategy for Non-experts to Create Customized Content with Models (2022, background): “…of thought (Wei et al., 2022) or scratchpad (Nye et al., 2021), majority voting (Wang et al., 2022b), reframing (Mishra et al., 2022a), least-to-most prompting (Zhou et al., 2022), question decomposition (Khot et al., 2020; Patel et al., 2022) have been shown to be effective across various tasks.”
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts (2021, background): “, 2020) and for sub-question generation (Khot et al., 2021)) by employing a variety of constraints as additional, domain-specific signals.”
Binding Language Models in Symbolic Languages (2022, methodology): “Khot et al. (2021) propose text module networks to solve complex tasks by decomposing them into simpler ones solvable by existing QA models and a symbolic calculator.”
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (2023, background): “[20] Tushar Khot, Daniel Khashabi, Kyle Richardson, Peter Clark, and Ashish Sabharwal.”
Successive Prompting for Decomposing Complex Questions (2022, methodology): “We adapt and extend the control codes introduced by text modular networks (TMNs, Khot et al., 2021) for training with our synthetic data.”
Temporal Reasoning on Implicit Events from Distant Supervision (2020, background): “, 2019) and other decomposition-based approaches (Talmor and Berant, 2018; Khashabi et al., 2018; Li et al., 2019; Wolfson et al., 2020; Khot et al., 2021).”
Is a Question Decomposition Unit All We Need? (2022, methodology): “True Answer : 10 Generated Answer : 14 The samples for QASC are provided without context.”