NumNet: Machine Reading Comprehension with Numerical Reasoning

By Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, Zhiyuan Liu. 2019

131 citations recorded by the source corpus.

Numerical reasoning, such as addition, subtraction, sorting and counting is a critical skill in human’s reading comprehension, which has not been well considered in existing machine reading comprehension (MRC) systems. To address this issue, we propose a numerical MRC model named as NumNet, which utilizes a numerically-aware graph neural network to consider the comparing information and performs numerical reasoning over numbers in the question and passage. Our system achieves an EM-score of 64.56% on the DROP dataset, outperforming all existing machine reading comprehension models by considering the numerical relations among numbers.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “…has also aimed to inject numeric reasoning abilities in language models in various ways, such as augmenting BERT with a predeﬁned set of executable operations (Andor et al., 2019), including a graph neural network (Ran et al., 2019), and using specialized training procedures (Pi˛ekos et al., 2021).”
Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, methodology): “Previous work has primarily focused on specialized approaches for improving reasoning (Andor et al., 2019; Ran et al., 2019; Geva et al., 2020; Pi˛ekos et al., 2021).”
Reasoning Like Program Executors (2022, methodology): “DROP♡ Specialized Models NumNet (Ran et al., 2019) 64.”
Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021, methodology): “…compare it to two state-ofthe-art black-box systems that not only lack interpretability but are also targeted towards specific datasets: NumNet+V2 (Ran et al., 2019) for DROP 19Since NMN-D focuses on a different subset, we report its score on the shared subset, on which MODULARQA achieves an F1…”
Question Directed Graph Attention Network for Numerical Reasoning over Text (2020, methodology): “NumNet (Ran et al., 2019) uses a numerically-aware graph neural network to encode numbers, which made further progress on the DROP dataset.”
Graph-to-Tree Learning for Solving Math Word Problems (2020, methodology): “…divided into statistical machine learning based (Kushman et al., 2014; Hosseini et al., 2014; Mitra and Baral, 2016; Roy and Roth, 2018; Zou and Lu, 2019a) and semantic parsing based (Shi et al., 2015; Koncel-Kedziorski et al., 2015; Roy and Roth, 2015; Huang et al., 2017; Zou and Lu, 2019b).”
Investigating the Limitations of Transformers with Simple Arithmetic Tasks (2021, background): “Others have proposed neural–symbolic hybrids, which are typically composed of a neural model to convert inputs to contiguous vector representations and a symbolic component that apply rules over these vectors (Ran et al., 2019).”
Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks (2020, methodology): “Examples include NumNet+v2 [Ran et al. , 2019], BERT Calculator [Andor et al. , 2019] and tag-based multi-span extraction model [Efrat et al. , 2019].”
Investigating the Limitations of the Transformers with Simple Arithmetic Tasks (2021, unknown): “Others have proposed neural–symbolic hybrids, which are typically composed of a neural model to convert inputs to contiguous vector representations and a symbolic component that apply rules over these vectors (Ran et al., 2019).”
Representing Numbers in NLP: a Survey and a Vision (2021, methodology): “(2020) find commonsense question answering to be one of the hardest among their Numbergame challenge, using the NumNetv2 model (Ran et al., 2019) which is commonly used for DROP question answering.”
NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks (2022, methodology): “We use two kinds of model architectures (i) Neuro-symbolic, a memory augmented novel architecture that extends Numnet+v2 (Ran et al., 2019) and (ii) End-to-end, GPT3 (Brown et al.”
Rethinking Search: Making Experts out of Dilettantes (2021, background): “While there has been recent work that investigates these aspects in the context of neural network models [72], it remains challenging to develop neural models that can reliably and accurately deliver on these types of reasoning”
OPERA: Operation-Pivoted Discrete Reasoning over Text (2022, methodology): “In the example, though NumNet+ and QDGAT correctly predict the answer type, the final answer is wrong.”
Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning (2022, methodology): “Firstly, we decide to choose the approach of explicitly modeling discrete operations, since existing NDR solutions have demonstrated its superiority (Dua et al., 2019; Ran et al., 2019; Herzig et al., 2020; Zhu et al., 2021).”
DAGN: Discourse-Aware Graph Network for Logical Reasoning (2021, methodology): “Inspired by previous graph-based models (Ran et al., 2019; Chen et al., 2020a), we also learn graph node representations to obtain higher-level features.”