LIMO: Less is More for Reasoning
By Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, Pengfei Liu. arXiv.org, 2025
487 citations recorded by the source corpus.
We challenge the prevailing assumption that complex reasoning in large language models (LLMs) necessitates massive training data. We demonstrate that sophisticated mathematical reasoning can emerge with only a few examples. Specifically, through simple supervised fine-tuning, our model, LIMO, achieves 63.3\% accuracy on AIME24 and 95.6\% on MATH500, surpassing previous fine-tuned models (6.5\% on AIME24, 59.2\% on MATH500) while using only 1\% of the training data required by prior approaches. Furthermore, LIMO exhibits strong out-of-distribution generalization, achieving a 45.8\% absolute improvement across diverse benchmarks, outperforming models trained on 100x more data. Synthesizing these findings, we propose the Less-Is-More Reasoning Hypothesis (LIMO Hypothesis): In foundation models where domain knowledge has been comprehensively encoded during pre-training, sophisticated reasoning can emerge through minimal but strategically designed demonstrations of cognitive processes. This hypothesis suggests that the threshold for eliciting complex reasoning is not dictated by task complexity but rather by two key factors: (1) the completeness of the model's pre-trained knowledge base and (2) the effectiveness of post-training examples in serving as"cognitive templates"that guide reasoning.
Subfields: Computational Theory and Mathematics, Management Science and Operations Research
View Paper: https://www.semanticscholar.org/paper/b62d0605137463ea591a0619840305cb98f6958f
Citations in the corpus
- Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models (2025, unknown): “Recent research has introduced innovative benchmarks and evaluation frameworks to systematically assess the reasoning capabilities of LLMs.”
- Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (2025, unknown): “Meanwhile, various methods [3–14] have been explored to generate high-quality complex CoT reasoning and further advance the field by optimizing reasoning pathways.”
- Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs (2025, unknown): “Reasoning Training Recent studies have suggested that training a robust reasoning model only requires a small amount of high-quality data, such as LIMO [YHX + 25] and S1K [MYS + 25].”