Towards Reasoning in Large Language Models: A Survey

By Jie Huang, Kevin Chen-Chuan Chang. 2022

934 citations recorded by the source corpus.

Reasoning is a fundamental aspect of human intelligence that plays a crucial role in activities such as problem solving, decision making, and critical thinking. In recent years, large language models (LLMs) have made significant progress in natural language processing, and there is observation that these models may exhibit reasoning abilities when they are sufficiently large. However, it is not yet clear to what extent LLMs are capable of reasoning. This paper provides a comprehensive overview of the current state of knowledge on reasoning in LLMs, including techniques for improving and eliciting reasoning in these models, methods and benchmarks for evaluating reasoning abilities, findings and implications of previous research in this field, and suggestions on future directions. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful discussion and future work.

Subfields: Artificial Intelligence, Large Language Models, Prompting & Reasoning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Prompting & Reasoning

Citations in the corpus

Augmented Language Models: a Survey (2023, methodology): “For the rest of this section, we discuss the works related to three popular paradigms for eliciting reasoning in LMs. Note that Huang and Chang (2022) propose a survey on reasoning in language models.”
A Comprehensive Overview of Large Language Models (2023, neutral): “A discussion on formatting in-context learning (ICL) templates is available in [86], [33], [26], [25].”
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning (2023, background): “Logical reasoning is a cognitive process that involves using evidence, arguments, and logic to arrive at conclusions or make judgments (Huang and Chang, 2022).”
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond (2023, background): “The arithmetic reasoning capability of LLMs benefits greatly from the scaling of model size [51].”
Reasoning with Language Model is Planning with World Model (2023, background): “More related works on world models and planning are discussed in the Appendix D. 3 Reasoning via Planning (RAP) In this section, we present the Reasoning via Planning (RAP) framework that enables LLMs to strategically plan a coherent reasoning trace for solving a wide range of reasoning tasks.”
When large language models meet personalization: perspectives of challenges and opportunities (2023, positive): “With the development of large language models, there is an observation that LLMs exhibit reasoning abilities [2, 161] when they are sufﬁciently large, which is fundamental for human 123 intelligence for decision-making and problem-solving.”
Making Large Language Models Better Reasoners with Alignment (2023, neutral): “Reasoning is a cognitive process that involves utilizing evidence to reach a well-founded conclusion (Qiao et al., 2023; Huang & Chang, 2023).”
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2022, background): “However, this may not align well with the actual quality of the rationale in many cases, as mentioned in Huang and Chang (2022). For example, a rationale that is correct for all but the last step (and hence derives the wrong final answer) would still be assigned a zero score, while a rationale that ”
Large Language Models: A Survey (2024, neutral): “As the field of LLMs is moving fast, with new findings, models and techniques being published in a matter of months or weeks [7], [8], [9], [10], [11], AI researchers and practitioners often find it challenging to figure out the best recipes to build LLM-powered AI systems for their tasks.”
R1-Onevision: Advancing Generalized Multimodal Reasoning Through Cross-Modal Formalization (2025, neutral): “The development of robust reasoning capabilities in Large Language Models (LLMs) has been a focal point of research [1, 11, 15, 21].”
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges (2024, background): “Huang et al., [18] presented a study on reasoning in LLMs that comprehensively summarizes the current state of LLMs’ reasoning capabilities.”
A survey on large language model based autonomous agents (2023, positive): “[69] presents the current state of research on LLMs’ reasoning abilities, exploring approaches to improve and evaluate their reasoning skills.”
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023, background): “We refer readers to the survey (Huang and Chang, 2022) for more related works.”
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (2023, background): “What’s especially striking is their ability to handle complex tasks through reasoning (Wei et al., 2022; Huang & Chang, 2023).”