MAWPS: A Math Word Problem Repository

By Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, Hannaneh Hajishirzi. 2016

523 citations recorded by the source corpus.

Recent work across several AI subdisciplines has focused on automatically solving math word problems. In this paper we introduce MAWPS, an online repository of Math Word Problems, to provide a unified testbed to evaluate different algorithms. MAWPS allows for the automatic construction of datasets with particular characteristics, providing tools for tuning the lexical and template overlap of a dataset as well as for filtering ungrammatical problems from web-sourced corpora. The online nature of this repository facilitates easy community contribution. At present, we have amassed 3,320 problems, including the full datasets used in several prominent works.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

PaLM: Scaling Language Modeling with Pathways (2022, methodology): “Using chain-of-thought prompting, we evaluate PaLM’s performance on the arithmetic datasets GSM8K (Cobbe et al., 2021), SVAMP (Patel et al., 2021), MAWPS (Koncel-Kedziorski et al., 2016), and AQuA (Ling et al., 2017), as well as the commonsense reasoning datasets CommonsenseQA (Talmor et al., 2019) ”
Chain of Thought Prompting Elicits Reasoning in Large Language Models (2022, neutral): “• Math Word Problem Repository (Koncel-Kedziorski et al., 2016): AddSub (Hosseini et al., 2014): https://www.cs.washington.edu/nlp/arithmetic ; MultiArith (Roy and Roth, 2015), license: CC BY 4.0.”
Toolformer: Language Models Can Teach Themselves to Use Tools (2023, methodology): “We test mathematical abilities on ASDiv (Miao et al., 2020), SVAMP (Patel et al., 2021) and the MAWPS benchmark (Koncel-Kedziorski et al., 2016).”
Self-Consistency Improves Chain of Thought Reasoning in Language Models (2022, methodology): “We use the Math Word Problem Repository (Koncel-Kedziorski et al., 2016), with the following datasets based on question types: AddSub (Hosseini et al.”
MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers (2021, background): “MAWPS-s en Single equation generation 1,987 - (Koncel-Kedziorski et al., 2016) Draw1K en Multiple equations generation 1,000 745 - (Upadhyay and Chang, 2017) Math23K zh Single equation generation 23,162 (Wang et al.”
Are NLP Models really able to Solve Simple Math Word Problems? (2021, methodology): “Koncel-Kedziorski et al. (2016) cu-”
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms (2019, background): “Most of these works have focused on algebra word problems, including MaWPS (Koncel-Kedziorski et al., 2016), Alg514 (Kushman et al.”
Large Language Models are Zero-Shot Reasoners (2022, background): “The first three are from the classic Math World Problem Repository [Koncel-Kedziorski et al., 2016], and the last three are from more recent benchmarks.”
Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction (2022, methodology): “We ﬁrst discuss the re-sults on MAWPS and Math23k, two datasets that are commonly used in previous research.”
Generate & Rank: A Multi-task Framework for Math Word Problems (2021, positive): “To evaluate the effectiveness of the proposed model, we conduct extensive experiments on the datasets of Math23K (Wang et al., 2017) and MAWPS (Koncel-Kedziorski et al., 2016).”
PAL: Program-aided Language Models (2022, methodology): “, 2020), and MAWPS (Koncel-Kedziorski et al., 2016); (2) symbolic reasoning (§4.”
How well do Computers Solve Math Word Problems? Large-Scale Dataset Construction and Evaluation (2016, methodology): “Alg514 is introduced in Kushman et al. (2014) and also used in Zhou et al.”
Annotating Derivations: A New Evaluation Strategy and Dataset for Algebra Word Problems (2016, methodology): “The datasets can also be used in evaluation test-beds, like the one proposed in (Koncel-Kedziorski et al., 2016).”
Birds Have Four Legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models (2020, background): “, 2020), thus making them able to answer math reasoning tasks such as MAWPS (Koncel-Kedziorski et al., 2016) and DROP (Dua et al.”
Mapping to Declarative Knowledge for Word Problem Solving (2017, methodology): “Following previous work (Roy and Roth, 2017), we compute two subsets of Aggregate comprising 756 problems each, using the MAWPS (Koncel-Kedziorski et al., 2016) 4 also shows that the other systems do not learn the right abstraction, even when trained on Aggregate.”
Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions (2019, methodology): “The experimental results show that the proposed model establishes the state-of-the-art performance on both Math23K and MAWPS datasets.”
Template-Based Math Word Problem Solvers with Recursive Neural Networks (2019, methodology): “We conduct experiments on two of the largest datasets for arithmetic word problems, in which Math23K contains 23 , 164 math problems and MAWPS contains 2 , 373 problems.”
Graph-to-Tree Learning for Solving Math Word Problems (2020, methodology): “transformer model (Koncel-Kedziorski et al., 2016; Cai and Lam, 2019).”