Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems
By Zhongli Li, Wenxuan Zhang, Chao Yan, Qingyu Zhou, Chao Li, Hongzhi Liu, Yunbo Cao. 2021
62 citations recorded by the source corpus.
Math Word Problem (MWP) solving needs to discover the quantitative relationships over natural language narratives. Recent work shows that existing models memorize procedures from context and rely on shallow heuristics to solve MWPs. In this paper, we look at this issue and argue that the cause is a lack of overall understanding of MWP patterns. We first investigate how a neural network understands patterns only from semantics, and observe that, if the prototype equations are the same, most problems get closer representations and those representations apart from them or close to other prototypes tend to produce wrong solutions. Inspired by it, we propose a contrastive learning approach, where the neural network perceives the divergence of patterns. We collect contrastive examples by converting the prototype equation into a tree and seeking similar tree structures. The solving model is trained with an auxiliary objective on the collected examples, resulting in the representations of problems with similar prototypes being pulled closer. We conduct experiments on the Chinese dataset Math23k and the English dataset MathQA. Our method greatly improves the performance in monolingual and multilingual settings.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Natural Language Processing
In Representation & Self-Supervised Learning
Citations in the corpus
- Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction (2022, methodology): “…Sun, 2019) 75.6 74.3 KA-S2T† (Wu et al., 2020) 76.3 - MultiE&D (Shen and Jin, 2020) 78.4 76.9 Graph2Tree (Zhang et al., 2020) 77.4 75.5 NeuralSymbolic (Qin et al., 2021) - 75.7 NUMS2T† (Wu et al., 2021) 78.1 - HMS (Lin et al., 2021) 76.1 - BERT-Tree (Li et al., 2021) 82 Table 4: Results on Math23k.”
- Investigating Math Word Problems using Pretrained Multilingual Language Models (2021, methodology): “Our contrastive learning approach differs with that of Li et al. (2022) in the following ways: (1) our method is focusing on cross-lingual setting that each pair of examples come from different languages, (2) we use batch-level contrastive training in consist with SimCLR.”
- A Survey of Deep Learning for Mathematical Reasoning (2022, unknown): “Outcome-based approaches acknowledge the potential incorrectness of an individual reasoning path, and instead use multiple reasoning paths (Wang et al., 2023; Li et al., 2022a).”