Point to the Expression: Solving Algebraic Word Problems Using the Expression-Pointer Transformer Model

By Bugeun Kim, Kyung Seo Ki, Donggeon Lee, Gahgene Gweon. 2020

44 citations recorded by the source corpus.

Solving algebraic word problems has recently emerged as an important natural language processing task. To solve algebraic word problems, recent studies suggested neural models that generate solution equations by using ‘Op (operator/operand)’ tokens as a unit of input/output. However, such a neural model suffered two issues: expression fragmentation and operand-context separation. To address each of these two issues, we propose a pure neural model, Expression-Pointer Transformer (EPT), which uses (1) ‘Expression’ token and (2) operand-context pointers when generating solution equations. The performance of the EPT model is tested on three datasets: ALG514, DRAW-1K, and MAWPS. Compared to the state-of-the-art (SoTA) models, the EPT model achieved a comparable performance accuracy in each of the three datasets; 81.3% on ALG514, 59.5% on DRAW-1K, and 84.5% on MAWPS. The contribution of this paper is two-fold; (1) We propose a pure neural model, EPT, which can address the expression fragmentation and the operand-context separation. (2) The fully automatic EPT model, which does not use hand-crafted features, yields comparable performance to existing models using hand-crafted features, and achieves better performance than existing pure neural models by at most 40%.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

Training Verifiers to Solve Math Word Problems (2021, background): “…specialized encoder-decoder architectures (Amini et al., 2019; Chiang and Chen, 2018; Xie and Sun, 2019; Chen et al., 2020; Li et al., 2020), with the strongest results often relying on large pretrained encoders from the BERT family (Chen et al., 2019; Kim et al., 2020; Liang et al., 2021).”
MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers (2021, methodology): “Over the last few years, there are a growing number of datasets (Kushman et al., 2014; Koncel-Kedziorski et al., 2016; Upadhyay and Chang, 2017; Huang et al., 2016; Wang et al., 2017; Qin et al., 2020; Miao et al., 2020; Patel et al., 2021) and deep learning-based methods that have been proposed to ”
Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022, methodology): “Numerous deep learning based methods have been proposed to tackle the MWP task, including Seq2Seq (Chiang and Chen, 2019; Li et al., 2019), Seq2Tree (Wang et al., 2019; Qin et al., 2020), and Pre-trained Language Models (Kim et al., 2020; Liang et al., 2021).”
Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems (2021, unknown): “Several works (Kim et al., 2020; Tan et al., 2021; Cobbe et al., 2021)”
A Survey of Deep Learning for Mathematical Reasoning (2022, unknown): “…ideas can be applied to math-related problems, and previous work has shown the promising performance of pre-trained language models in answering math word problems (Kim et al., 2020), assisting with theorem proving (Wu et al., 2022b), as well as solving other mathematical tasks (Charton, 2022).”
Investigating Math Word Problems using Pretrained Multilingual Language Models (2021, methodology): “For example, EPT (Kim et al., 2020) adopts ALBERT (Lan et al., 2020) as the encoder for its sequence-to-sequence module.”
FinQA: A Dataset of Numerical Reasoning over Financial Data (2021, methodology): “Existing models include (Kim et al., 2020; Chen et al., 2020a,d), etc. Financial NLP. Financial NLP has become one of the major application domains attracting growing attentions.”