Tree of Thoughts: Deliberate Problem Solving with Large Language Models
By Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, T. Griffiths, Yuan Cao, Karthik Narasimhan. 2023
4,499 citations recorded by the source corpus.
Language models are increasingly being deployed for general problem solving across a wide range of tasks, but are still confined to token-level, left-to-right decision-making processes during inference. This means they can fall short in tasks that require exploration, strategic lookahead, or where initial decisions play a pivotal role. To surmount these challenges, we introduce a new framework for language model inference, Tree of Thoughts (ToT), which generalizes over the popular Chain of Thought approach to prompting language models, and enables exploration over coherent units of text (thoughts) that serve as intermediate steps toward problem solving. ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices. Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. Code repo with all prompts: https://github.com/princeton-nlp/tree-of-thought-llm.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Large Language Models, Prompting & Reasoning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Percy Liang
- Luke S. Zettlemoyer
- Quoc V. Le
- Graham Neubig
- Yoshua Bengio
- Yejin Choi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Ming-Hsuan Yang
- Jitendra Malik
- Andrew Zisserman
- Alan L. Yuille
In Large Language Models
- Jianfeng Gao
- Jie Tang
- Zhiyuan Liu
- Minlie Huang
- Quoc V. Le
- Richard Socher
- Jason Aaron Edward Weston
- Mohit Bansal
In Prompting & Reasoning
Citations in the corpus
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024, methodology): “In the future, we will explore our RL pipeline on out-of-distribution question prompts, in conjunction with advanced sampling (decoding) strategies , like those based on tree-search methods (Yao et al., 2023).”
- Understanding the planning of LLM agents: A survey (2024, methodology): “Benefiting from the tree architecture, Tree-of-Thought (ToT) [Yao et al. , 2023] supports tree search algorithms, such as conventional BFS and DFS.”
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (2023, background): “…thought of CoT has been improved along different dimensions, including Auto-CoT (Zhang et al., 2022), Complex-CoT (Fu et al., 2023), Self-Consistency (Wang et al., 2023c), Zero-Shot-CoT (Kojima et al., 2022), Iter-CoT (Sun et al., 2023b), ToT (Yao et al., 2023), GoT (Besta et al., 2023) and so on.”
- A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications (2024, neutral): “To address these challenges, Yu et al. [2023] introduced a novel approach to improve RALMs robustness by handling noisy, irrelevant documents and accurately addressing unknown scenarios.”
- Large Language Model based Multi-Agents: A Survey of Progress and Challenges (2024, background): “Hence, LLM-based agent has been studied and rapidly developed to understand and generate human-like instructions, facilitating sophisticated interactions and decision-making in a wide range of contexts [Yao et al. , 2023; Shinn et al. , 2023; Li et al. , 2023d].”
- Knowledge Graph Prompting for Multi-Document Question Answering (2023, background): “Recent advancements explore different prompting strategies to enhance LLMs’ reasoning capabilities (Wei et al. 2022; Yao et al. 2023).”
- Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024, methodology): “Prompting techniques like Chain-of-Thought [69] and Tree-of-Thought [74] have been applied to improve knowledge recall and reasoning [67].”
- Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities (2024, methodology): “Feedback acquisition follows, where the LLM evaluates the plan’s efficacy through internal assessments or external signals, refining the strategy based on outcomes from different environments [74].”
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents (2023, neutral): “…mechanisms that equip agents with capabilities for reflection (Yao et al., 2023b; Shinn et al., 2023), task decomposition (Wei et al., 2022b; Yao et al., 2023a), and tool utilization/creation (Schick et al., 2023b; Qin et al., 2023a;b; Qian et al., 2023b) capabilities, thereby enabling…”
- Bilingual language model for protein sequence and structure (2024, neutral): “Our proof-of-principle solution rendering protein structures directly amenable to LLMs will benefit from future NLP improvements, in particular, from better sampling from conditional language models [66].”
- Scaling Relationship on Learning Mathematical Reasoning with Large Language Models (2023, background): “Many recent works are focusing on using different prompts (Wei et al., 2022b; Yao et al., 2023) or ensembling / reranking multiple times of inferences (Cobbe et al., 2021; Uesato et al., 2022; Wang et al., 2023; Lightman et al., 2023) to improve models’ reasoning performances.”
- LIMO: Less is More for Reasoning (2025, neutral): “This can be achieved by augmenting LLMs with methods such as parallel sampling (Brown et al., 2024; Wang et al., 2022; Li et al., 2022) or symbolic tree search (Hao et al., 2023; Chen et al., 2024; Yao et al., 2023) to enhance reasoning ability.”
- ReTool: Reinforcement Learning for Strategic Tool Use in LLMs (2025, neutral): “Recent advancements in large language models (LLMs) [3, 4, 9, 12, 19, 20, 25, 27, 29, 31] indicate significant progress toward cognitive abilities similar to human metacognition through Chain-of-Thought (CoT) prompting.”
- Mathematical Capabilities of ChatGPT (2023, methodology): “…we have deliberately chosen to avoid more complex types of prompt engineering mechanics, such as using Chain-of-Thoughts [61], or Tree-of-Thoughts [62] because 1) the main focus of the paper is the standard performance of the tested language models, not their the performance augmented by various…”
- Large Language Models as Commonsense Knowledge for Large-Scale Task Planning (2023, methodology): “LLMs have been used as a few-shot policy for language-conditioned task planning [18, 2, 19], or a policy for multi-step reasoning problems [37, 41].”