Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning

By Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, Song-Chun Zhu. 2021

520 citations recorded by the source corpus.

Geometry problem solving has attracted much attention in the NLP community recently. The task is challenging as it requires abstract problem understanding and symbolic reasoning with axiomatic knowledge. However, current datasets are either small in scale or not publicly available. Thus, we construct a new large-scale benchmark, Geometry3K, consisting of 3,002 geometry problems with dense annotation in formal language. We further propose a novel geometry solving approach with formal language and symbolic reasoning, called Interpretable Geometry Problem Solver (Inter-GPS). Inter-GPS first parses the problem text and diagram into formal language automatically via rule-based text parsing and neural object detecting, respectively. Unlike implicit learning in existing methods, Inter-GPS incorporates theorem knowledge as conditional rules and performs symbolic reasoning step by step. Also, a theorem predictor is designed to infer the theorem application sequence fed to the symbolic solver for the more efficient and reasonable searching path. Extensive experiments on the Geometry3K and GEOS datasets demonstrate that Inter-GPS achieves significant improvements over existing methods. The project with code and data is available at https://lupantech.github.io/inter-gps.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers (2021, background): “In addition, weakly supervised learning for MWP solving (Hong et al., 2021; Chatterjee et al., 2021) and supervised learning for geometric problem solving (Lu et al., 2021; Chen et al., 2021) have recently attracted much researchers’ attention.”
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering (2022, neutral): “Different from existing datasets [49, 16, 51, 30, 23], S CIENCE QA covers diverse topics across three subjects: natural science, social science, and language science.”
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models (2023, methodology): “In the domain of mathematical reasoning, an interpretable problem solver has been developed to incorporate theorem knowledge as conditional rules and perform symbolic reasoning step by step [26].”
What matters when building vision-language models? (2024, unknown): “TabMWP (Lu et al., 2023) 22,729 23,059 1,948,166 2.49% TAT-QA (Zhu et al., 2021) 2,199 13,215 283,776 2.18% HiTab (Cheng et al., 2022) 2,500 7,782 351,299 1.80% MultiHiertt (Zhao et al., 2022) 7,619 7,830 267,615 1.37% FinQA (Chen et al., 2021) 5,276 6,251 242,561 0.99% WikiSQL (Zhong et al., 2017) ”
Large Language Models for Mathematical Reasoning: Progresses and Challenges (2024, unknown): “An example can be seen as follows and Table 3 (Seo et al., 2015) 186 G EO S++ (Sachan et al., 2017) 1.4K G EO S-OS (Sachan and Xing, 2017) 2.2K G EOMETRY 3K (Lu et al., 2021) 3K G EO QA (Chen et al., 2021a) 5K U NI G EO (Chen et al., 2022) 14.5K”
Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models (2024, methodology): “To evaluate the effectiveness of S KETCHPAD , we use the problems from the Geometry3K dataset [32].”
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models (2024, unknown): “Additionally, we employ Geometry3K (Lu et al., 2021a) to enhance the model’s geometry problem-solving abilities.”
SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models (2023, methodology): “The data is manually collected by human annotators using a web-based annotation tool (Lu et al., 2021a), whose user interface is shown in Appendix A.3.”
How far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites (2024, unknown): “These datasets span multiple tasks, task ratio dataset Laion-EN (en) [89], Laion-ZH (zh) [89], COYO (zh) [12], Captioning 53.9% GRIT (zh) [86], COCO (en) [19], TextCaps (en) [95] Objects365 (en&zh) [93], GRIT (en&zh) [86], Detection 5.2% All-Seeing (en&zh) [114] Wukong-OCR (zh) [29], LaionCOCO-OCR (”
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (2025, unknown): “During GRPO process, we mix the math datasets of We-Math [39], MathVision [40], Polymath [41], SceMQA [42], Geometry3K [43] as our RL training data.”
PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning (2021, background): “Also, while previous datasets emphasize reasoning based on spatial relationships, humans possess diverse reasoning abilities, including analogy [62], geometry[48, 40], arithmetic[63, 20, 21], and physics [13].”
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models (2025, unknown): “2 [124], which cover a wide range of domains, including general visual question answering (VQA) [43, 50, 90, 83, 127, 126], science [57, 16, 82], chart [91, 54, 11], mathematics [72, 104, 10, 81, 55, 40, 147, 106], OCR [92, 107, 9, 49, 96], and document [24].”
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts (2023, background): “…several works that focus on evaluating specific skills of the LMMs beyond natural scenes, such as abstract scenes and shapes) (Antol et al., 2015; Lu et al., 2021b; Ji et al., 2022), geometry diagrams (Seo et al., 2015; Lu et al., 2021a; Chen et al., 2022a; Cao & Xiao, 2022), figures and charts…”
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems (2024, methodology): “For assessing multimodal capability, works such as Geome-try3K (Lu et al., 2021), GeoQA (Chen et al., 2021), GeoQA+ (Cao and Xiao, 2022), and Uni-Geo (Chen et al., 2022) have employed multimodal information for tackling geometric problems, integrating natural language descriptions with diagrams.”
A Survey of Deep Learning for Mathematical Reasoning (2022, unknown): “In recent years, there has been growing interest in multi-modal mathematical reasoning, which involves using multiple sources of information, such as text, tables, natural images, and diagrams (Ka-hou et al., 2018; Kaﬂe et al., 2018; Lu et al., 2021b, 2022b).”