Neural Module Networks for Reasoning over Text

By Nitish Gupta, Kevin Lin, Dan Roth, Sameer Singh, Matt Gardner. 2019

134 citations recorded by the source corpus.

Answering compositional questions that require multiple steps of reasoning against text is challenging, especially when they involve discrete, symbolic operations. Neural module networks (NMNs) learn to parse such questions as executable programs composed of learnable modules, performing well on synthetic visual QA domains. However, we find that it is challenging to learn these models for non-synthetic questions on open-domain text, where a model needs to deal with the diversity of natural language and perform a broader range of reasoning. We extend NMNs by: (a) introducing modules that reason over a paragraph of text, performing symbolic reasoning (such as arithmetic, sorting, counting) over numbers and dates in a probabilistic and differentiable manner; and (b) proposing an unsupervised auxiliary loss to help extract arguments associated with the events in text. Additionally, we show that a limited amount of heuristically-obtained question program and intermediate module output supervision provides sufficient inductive bias for accurate learning. Our proposed model significantly outperforms state-of-the-art models on a subset of the DROP dataset that poses a variety of reasoning challenges that are covered by our modules.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Reasoning Like Program Executors (2022, background): “(Andreas et al., 2016), reading comprehension (Gupta et al., 2019; Khot et al., 2021), knowledge base question answering (Ren et al., 2021) and 3D rendering (Tian et al., 2019).”
Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction (2022, methodology): “We design differentiable modules (Andreas et al., 2016; Gupta et al., 2020) in our model (§3.2) to perform reasoning among the quantities.”
Explainable Multi-hop Verbal Reasoning Through Internal Monologue (2021, background): “Another direction are the neural modular networks, where what components to use are determined dynamically for each question (Gupta et al., 2019; Jiang and Bansal, 2019).”
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning (2022, background): “Another attempt to bridge the gap comes from the neurosymbolic perspective (Gupta et al., 2019; Hudson and Manning, 2019; Mao et al., 2019; Yi et al., 2018).”
PAL: Program-aided Language Models (2022, background): “Andor et al. (2019) generate task-specific arithmetic operations for reading comprehension tasks; Gupta et al. (2019) design neural modules such as count to deal with arithmetic operations.”
Learning from Task Descriptions (2020, background): “Attempts to generalize semantic parsing to more open domains are still nascent (Chen et al., 2020; Gupta et al., 2020).”
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, methodology): “(15)Previous modular systems (Gupta et al., 2020) have targeted even smaller subsets to develop modular approaches.”
Question Directed Graph Attention Network for Numerical Reasoning over Text (2020, background): “, 2019b), NMNs (Gupta et al., 2020), NumNet (Ran et al.”
Reason first, then respond: Modular Generation for Knowledge-infused Dialogue (2021, background): “The approach of text modular networks promises more interpretable answers to multi-hop questions (Khot et al., 2020; Jiang and Bansal, 2019; Gupta et al., 2020).”
Decomposed Prompting: A Modular Approach for Solving Complex Tasks (2022, methodology): “…for Multi-Step Reasoning Our work follows a long literature in NLP on neural modular modeling architectures (Andreas et al., 2016; Talmor & Berant, 2018; Jiang & Bansal, 2019; Gupta et al., 2020; Perez et al., 2020; Khot et al., 2021; Levine et al., 2022) for question-answering and other tasks.”
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills (2021, methodology): “Reasoning skills in DROP To check which reasoning skills PReasM has, we use a proposed split of a subset of DROP to reasoning skills (Gupta et al., 2020a).”
Program Enhanced Fact Verification with Verbalization and Graph Attention Network (2020, methodology): “However, such methods tend to fail when veriﬁcation requires the joint modelling of both symbolic operations and language inference (Gupta et al., 2020) such as the case depicted in Fig.”
Temporal Reasoning on Implicit Events from Distant Supervision (2020, background): “, 2012), as well as neural module networks (Andreas et al., 2016; Gupta et al., 2019) and other decomposition-based approaches (Talmor and Berant, 2018; Khashabi et al.”
A Reinforcement Learning Environment for Mathematical Reasoning via Program Synthesis (2021, unknown): “We think it would be desirable to train neural networks to make use of existing programs for computing various mathematical operations [4][5].”
Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text (2021, background): “, 2021) and neural module networks (Gupta et al., 2019) also have been attempted to partly imitate the human reasoning process.”
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap (2023, background): “More recent research aims to develop “explainable architectures” with representations meaningful to people (e.g., Gupta et al., 2019; Yi et al., 2018).”
Post-hoc Interpretability for Neural NLP: A Survey (2021, background): “However, even intrinsically interpretable methods such as Attention and Neural Modular Networks have been shown to not reflect the model [54, 112].”
Exploring End-to-End Differentiable Natural Logic Modeling (2020, methodology): “We explore a framework in which module networks (Andreas et al., 2016; Gupta et al., 2020) are leveraged to model the natural logic operations, which is enhanced with a memory module component to MacCartney and Manning (2009).”