MathDQN: Solving Arithmetic Word Problems via Deep Reinforcement Learning

By Lei Wang, Dongxiang Zhang, Lianli Gao, Jingkuan Song, Long Guo, Heng Tao Shen. 2018

122 citations recorded by the source corpus.

Designing an automatic solver for math word problems has been considered as a crucial step towards general AI, with the ability of natural language understanding and logical inference. The state-of-the-art performance was achieved by enumerating all the possible expressions from the quantities in the text and customizing a scoring function to identify the one with the maximum probability. However, it incurs exponential search space with the number of quantities and beam search has to be applied to trade accuracy for efficiency. In this paper, we make the first attempt of applying deep reinforcement learning to solve arithmetic word problems. The motivation is that deep Q-network has witnessed success in solving various problems with big search space and achieves promising performance in terms of both accuracy and running time. To fit the math problem scenario, we propose our MathDQN that is customized from the general deep reinforcement learning framework. Technically, we design the states, actions, reward function, together with a feed-forward neural network as the deep Q-network. Extensive experimental results validate our superiority over state-of-the-art methods. Our MathDQN yields remarkable improvement on most of datasets and boosts the average precision among all the benchmark datasets by 15\%.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

NumNet: Machine Reading Comprehension with Numerical Reasoning (2019, neutral): “Wang et al. (2018) leveraged deep Q-network to solve the AWP problems, achieving a good balance between effectiveness and efﬁciency.”
MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers (2021, methodology): “The models include Seq2seq, Seq2tree, Graph2tree, and commonly-used non-pretrained (AttSeq2Seq (Bahdanau et al., 2014), LSTMVAE (Zhang et al., 2016), and Trans-former (Vaswani et al., 2017)) and pretrained models (GPT-2 (Radford et al., 2019), BERTGen (De-vlin et al., 2018), and RoBERTaGen (Liu et a”
Measuring and Improving BERT’s Mathematical Abilities by Predicting the Order of Reasoning. (2021, methodology): “various approaches, more traditional symbolic methods, neural networks, and neuro-symbolic methods. (Bobrow,1964;Charniak,1969;Shi et al.,2015;Ling et al.,2017;Amini et al.,2019;Parisotto et al.,2016;Wang et al., 2018;Zou and Lu,2019) as well as datasets (Ling et al.,2017;Amini et al.,2019;Huang et ”
Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension (2019, methodology): “Compared to other more general systems, our model outperforms the deep reinforcement learning based approach of Wang et al. (2018).”
Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems (2018, background): “…2015; Roy and Roth, 2015; Koncel-Kedziorski et al., 2015; Roy et al., 2016; Upadhyay et al., 2016; Upadhyay and Chang, 2017; Roy and Roth, 2018; Wang et al., 2018 In terms of how to learn the model for solving math word problems, the prior work can be categorized into three: 1) the ﬁrst…”
A Goal-Driven Tree-Structured Neural Model for Math Word Problems (2019, methodology): “Researchers have proposed several approaches [Ling et al. , 2017; Wang et al. , 2017; Huang et al. , 2018; Wang et al. , 2018a] to solving MWPs based on the Seq2Seq model [Sutskever et al. , 2014].”
Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions (2019, methodology): “, we normalize the equations to postfix expressions following the rules in Wang et al. (2018a), ensuring that every problem is corresponding to a unique equation.”
Template-Based Math Word Problem Solvers with Recursive Neural Networks (2019, methodology): “(Wang et al. 2018b) modeled the math expression as a tree structure and made the ﬁrst attempt to apply deep reinforcement learning for iterative tree construction.”
A Knowledge-Aware Sequence-to-Tree Network for Math Word Problem Solving (2020, methodology): “Wang et al. (2018b) incorporated reinforcement learning into the model to construct a math expression step by step.”
HMS: A Hierarchical Solver with Dependency-Enhanced Understanding for Math Word Problem (2021, methodology): “Better results can be achieved with better modeling and representations of complex problem structures (Wang et al. 2018a).”
Translating a Math Word Problem to an Expression Tree (2018, background): “Wang et al. (2018) then make the first attempt of applying deep reinforcement learning to arithmetic word problem solving.”
Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems (2020, methodology): “Recently, deep learning has opened a new direction towards automatic MWPs solving (Wang et al., 2017; Huang et al., 2018; Wang et al., 2018b, 2019; Xie and Sun, 2019; Chiang and Chen, 2019).”
An Overview of Natural Language State Representation for Reinforcement Learning (2020, background): “Hierarchical network representations combining word level and sentence level encodings were proposed, as in text classi-ﬁcation (Zhang et al., 2018).”
A Survey of Deep Learning for Mathematical Reasoning (2022, unknown): “MathDQN (Wang et al., 2018b) MWP Search DQN - 7 RL with a deep Q-network (DQN) DDT (Meng and Rumshisky, 2019) MWP Generation Transformer Trm Trm 4 A Transformer-based model DeepMath (Irving et al.”
The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers (2018, methodology): “As shown in Table 5, a binary indicator to determine whether a quantity refers to a rate is adopted in many solvers [26] [28] [15] [40] [45].”
Graph-to-Tree Neural Networks for Learning Structured Input-Output Translation with Applications to Semantic Parsing and Math Word Problem (2020, methodology): “To resolve the equation duplication issues, various equation normalization methods are proposed in (Wang et al., 2018a, 2019) to generate a unique expression tree with the cost of losing the understanding of problem-solving steps in equation expressions.”
Teacher-Student Networks with Multiple Decoders for Solving Math Word Problem (2020, methodology): “The solution expression is represented by either a sequential [ Wang et al. , 2017; Wang et al. , 2018a; Chiang and Chen, 2019] or a tree structure [Liu et al. , 2019; Xie and Sun, 2019].”
Neural Math Word Problem Solver with Reinforcement Learning (2018, unknown): “Wang et al. (2018) extract features for quantity pairs and uses a reinforcement framework to construct an equation tree with the constraint of one unknown variable.”
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks (2021, background): “…methods (Shi et al., 2015; Koncelkedziorski et al., 2015; Huang et al., 2017; Liang et al., 2018a), to deep learning meth-ods (Ling et al., 2017; Wang et al., 2017, 2018b; Huang et al., 2018; Wang et al., 2018a; Xie and Sun, 2019; Wang et al., 2019; Zhang et al., 2020a,b; Qin et al., 2020; Shen…”
Tree-structured Decoding for Solving Math Word Problems (2019, methodology): “Wang et al. (2018a) introduced equation normalization techniques that leverage the duplicated template problem.”