Large Language Models for Mathematical Reasoning: Progresses and Challenges

By Janice Ahn, Rishu Verma, Renze Lou, Dingxiao Liu, Rui Zhang, Wenpeng Yin. Conference of the European Chapter of the Association for Computational Linguistics, 2024

349 citations recorded by the source corpus.

Mathematical reasoning serves as a cornerstone for assessing the fundamental cognitive capabilities of human intelligence. In recent times, there has been a notable surge in the development of Large Language Models (LLMs) geared towards the automated resolution of mathematical problems. However, the landscape of mathematical problem types is vast and varied, with LLM-oriented techniques undergoing evaluation across diverse datasets and settings. This diversity makes it challenging to discern the true advancements and obstacles within this burgeoning field. This survey endeavors to address four pivotal dimensions: i) a comprehensive exploration of the various mathematical problems and their corresponding datasets that have been investigated; ii) an examination of the spectrum of LLM-oriented techniques that have been proposed for mathematical problem-solving; iii) an overview of factors and concerns affecting LLMs in solving math; and iv) an elucidation of the persisting challenges within this domain. To the best of our knowledge, this survey stands as one of the first extensive examinations of the landscape of LLMs in the realm of mathematics, providing a holistic perspective on the current state, accomplishments, and future challenges in this rapidly evolving field.

View Paper: https://www.semanticscholar.org/paper/42445823fb0156afddc8c72eaa5ee81dded5b965

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Tal Remez
2. Yossi Adi
3. Itai Gat
4. Gabriel Synnaeve
5. Felix Kreuk
6. Neta Shaul
7. Yaron Lipman
8. Ricky T. Q. Chen

Citations in the corpus

• WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct (2023, unknown): “However, these open models still struggle with the scenarios which require complex multi-step quantitative reasoning, such as solving mathematical and science challenges (Ahn et al., 2024; Long et al., 2024).”
• R1-Onevision: Advancing Generalized Multimodal Reasoning Through Cross-Modal Formalization (2025, unknown): “The development of robust reasoning capabilities in Large Language Models (LLMs) has been a focal point of research [1, 11, 15, 21].”
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning (2025, unknown): “(3) Fine-Tuning-Based Methods : Supervised fine-tuning (SFT) (Chung et al., 2024), RL-based fine-tuning without a search engine (R1) (Guo et al., 2025) and rejection sampling (Ahn et al., 2024) with a search engine.”
• Discrete Flow Matching (2024, background): “…as, code (Roziere et al., 2023; Li et al., 2023; Chen et al., 2021), biology (Zhang et al., 2024; Ferruz and Höcker, 2022; Madani et al., 2023), math (Romera-Paredes et al., 2024; Imani et al., 2023; Ahn et al., 2024), audio (Kreuk et al., 2022; Copet et al., 2024; Hassid et al., 2024) and more.”