ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
By Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen. 2023
314 citations recorded by the source corpus.
Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools (e.g., computation libraries and symbolic solvers), thereby amalgamating the analytical prowess of language and the computational efficiency of tools. To train ToRA, we curate interactive tool-use trajectories on mathematical datasets, apply imitation learning on the annotations, and propose output space shaping to further refine models' reasoning behavior. As a result, ToRA models significantly outperform open-source models on 10 mathematical reasoning datasets across all scales with 13%-19% absolute improvements on average. Notably, ToRA-7B reaches 44.6% on the competition-level dataset MATH, surpassing the best open-source model WizardMath-70B by 22% absolute. ToRA-Code-34B is also the first open-source model that achieves an accuracy exceeding 50% on MATH, which significantly outperforms GPT-4's CoT result, and is competitive with GPT-4 solving problems with programs. Additionally, we conduct a comprehensive analysis of the benefits and remaining challenges of tool interaction for mathematical reasoning, providing valuable insights for future research.
Subfields: Artificial Intelligence, Developmental and Educational Psychology, Prompting & Reasoning, LLM Agents & Tool Use
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Developmental and Educational Psychology
In Prompting & Reasoning
Citations in the corpus
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024, methodology): “After pre-training, we apply mathematical instruction tuning to DeepSeekMath-Base with chain-of-thought (Wei et al., 2022), program-of-thought (Chen et al., 2022; Gao et al., 2023), and tool-integrated reasoning (Gou et al., 2023) data.”
- DeepSeek LLM: Scaling Open-Source Language Models with Longtermism (2024, methodology): “We have conducted a comparative analysis of our model against various baseline models of different sizes, namely Qwen 72B Chat (Bai et al., 2023), ChatGLM3 (Du et al., 2022), Baichuan2 (Yang et al., 2023), and Yi-34B Chat.”
- A Survey on Knowledge Distillation of Large Language Models (2024, background): “LawyerLLaMA (Huang et al., 2023b), LawGPT (Cui et al., 2023b), Fuzi (Wu et al., 2023d) Medical and Healthcare Huatuogpt (Zhang et al., 2023c), Huatuogpt-II (Chen et al., 2023d), Doctorglm (Xiong et al., 2023), Alpacare (Zhang et al., 2023f), Huatuo (Wang et al., 2023a), ChatDoctor (Li et al., 2023i)”
- TÜLU 3: Pushing Frontiers in Open Language Model Post-Training (2024, unknown): “22 Furthermore, training models to use tools is a natural way to dramatically improve their reasoning and mathematical skills [Gou et al., 2024], rather than trying to accomplish everything ‘in the weights.’”
- Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations (2023, background): “2) fine-tuning : The fine-tuning meth-ods (Yu et al., 2023b; Luo et al., 2023; Yue et al., 2023; Wang et al., 2023b; Gou et al., 2023) can also enhance the mathematical reasoning ability of LLMs.”
- A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, unknown): “By leveraging these datasets, developers can tailor memory capabilities to better handle task-specific challenges, resulting in more efficient and responsive LLM-based agents.”
- rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking (2025, unknown): “Advancements in LLM math reasoning have largely relied on curating high-quality CoT data, with most leading approaches being GPT-distilled, using frontier models like GPT-4 for synthesis [Wang et al., 2024b, Gou et al., 2023, Luo et al., 2023].”