Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning

By Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, Ashwin Kalyan. 2022

463 citations recorded by the source corpus.

Mathematical reasoning, a core ability of human intelligence, presents unique challenges for machines in abstract thinking and logical reasoning. Recent large pre-trained language models such as GPT-3 have achieved remarkable progress on mathematical reasoning tasks written in text form, such as math word problems (MWP). However, it is unknown if the models can handle more complex problems that involve math reasoning over heterogeneous information, such as tabular data. To fill the gap, we present Tabular Math Word Problems (TabMWP), a new dataset containing 38,431 open-domain grade-level problems that require mathematical reasoning on both textual and tabular data. Each question in TabMWP is aligned with a tabular context, which is presented as an image, semi-structured text, and a structured table. There are two types of questions: free-text and multi-choice, and each problem is annotated with gold solutions to reveal the multi-step reasoning process. We evaluate different pre-trained models on TabMWP, including the GPT-3 model in a few-shot setting. As earlier studies suggest, since few-shot GPT-3 relies on the selection of in-context examples, its performance is unstable and can degrade to near chance. The unstable issue is more severe when handling complex problems like TabMWP. To mitigate this, we further propose a novel approach, PromptPG, which utilizes policy gradient to learn to select in-context examples from a small amount of training data and then constructs the corresponding prompt for the test example. Experimental results show that our method outperforms the best baseline by 5.31% on the accuracy metric and reduces the prediction variance significantly compared to random selection, which verifies its effectiveness in selecting in-context examples.

Subfields: Natural Language Processing, Prompting & Reasoning, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Prompting & Reasoning

In Reinforcement Learning

Citations in the corpus

Multimodal Chain-of-Thought Reasoning in Language Models (2023, neutral): “However, existing studies related to CoT reasoning are largely isolated in the language modality (Wang et al., 2022b; Zhou et al., 2022; Lu et al., 2022b; Fu et al., 2022), with little consideration of multimodal scenarios.”
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering (2022, background): “Recent research work [4, 38, 34] has shown that GPT-3 [4] can perform various tasks when provided with in-context examples in a standard prompt.”
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks (2022, methodology): “On TabWMP (Lu et al., 2022), the prior best result is achieved by Dynamic Prompt Learning (Lu et al.”
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (2023, methodology): “Table 5: QA accuracy (%) on the test set of TabMWP [28].”
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites (2024, unknown): “Detection and grounding tasks utilize datasets like Ob-jects365 [93], GRIT [86], and All-Seeing [114], making up 5.2%.”
Large Language Model Guided Tree-of-Thought (2023, background): “PromptPG [24] proposes an approach that can learn to select in-context examples from a small amount of training data via policy gradient for prompt learning.”
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models (2024, background): “…from 165k images by prompting 21], Infograph-icVQA train [22], AI2D train (transparent and opaque label boxes) [13], A-OKVQA train [28], Android-Control train [14], ScienceQA train [16], TabMWP train [17], ST-VQA train [6], TallyQA train [3], DVQA train [11], FigureQA train [12], and train [23].”
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts (2023, background): “Our analysis highlights that hallucination is a major source of errors in the generative foundation models (Lu et al., 2023c; Ji et al., 2023).”
SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models (2023, methodology): “Recent benchmarks focus on assessing problem-solving skills of LLMs, particularly in scientific and mathematical domains [31, 11, 30, 49, 33, 8, 14, 16].”
What matters when building vision-language models? (2024, unknown): “TabMWP (Lu et al., 2023) 22,729 23,059 1,948,166 2.49% TAT-QA (Zhu et al., 2021) 2,199 13,215 283,776 2.18% HiTab (Cheng et al., 2022) 2,500 7,782 351,299 1.80% MultiHiertt (Zhao et al., 2022) 7,619 7,830 267,615 1.37% FinQA (Chen et al., 2021) 5,276 6,251 242,561 0.99% WikiSQL (Zhong et al., 2017)…”
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models (2024, unknown): “In this part, we will briefly review studies with these two types of generators: parameter-accessible (white-box) and parameter-inaccessible (black-box).”
Decomposition Enhances Reasoning via Self-Evaluation Guided Decoding (2023, unknown): “The results are consistent across other datasets, including AQuA (55.9% vs. 54.7%), SVAMP (89.6% vs. 84.1%), ASDiv (84.9% vs. 84.1%), and TabMWP (79.1% vs. 80.6%).”
TheoremQA: A Theorem-driven Question Answering dataset (2023, background): “(Patel et al., 2021a) Arithm/Algebra Elementary School ASDiv - Math23K (Wang et al., 2017) Algebra Elementary School Internet - TabMWP (Lu et al., 2023b) Arithm/Algebra Elem.”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, methodology): “, 2021), and TabMWP (Lu et al., 2023), we utilize the official test split.”
ToolQA: A Dataset for LLM Question Answering with External Tools (2023, methodology): “Besides, the mathematical reasoning abilities of external calculators and Python interpreters are evaluated using computation-intensive QA datasets [11, 33].”