ReAct: Synergizing Reasoning and Acting in Language Models
By Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao. International Conference on Learning Representations, 2022
8,935 citations recorded by the source corpus.
While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information. We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines, as well as improved human interpretability and trustworthiness over methods without reasoning or acting components. Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generates human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples. Project site with code: https://react-lm.github.io
Subfields: Artificial Intelligence, Prompting & Reasoning, LLM Agents & Tool Use
View Paper: https://arxiv.org/abs/2210.03629
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Jianfeng Gao
- Percy Liang
- Luke S. Zettlemoyer
- Graham Neubig
- Yejin Choi
- Quoc V. Le
- Yoshua Bengio
In Prompting & Reasoning
- Luke S. Zettlemoyer
- Minlie Huang
- Denny Zhou
- Zhiyuan Liu
- Mohit Bansal
- Michael Lewis
- Tushar Khot
- Ashish Sabharwal
In LLM Agents & Tool Use
- Stephen Marcus McAleer
- Pierre Baldi
- Geunwoo Kim
- Andy Zeng
- Aleksandra Faust
- Izzeddin Gur
- Percy Liang
- Timothy P. Lillicrap
Citations in the corpus
- Toolformer: Language Models Can Teach Themselves to Use Tools (2023, background): “…ability to use external tools such as search engines (Komeili et al., 2022; Thoppilan et al., 2022; Lazaridou et al., 2022; Shuster et al., 2022; Yao et al., 2022), web browsers (Nakano et al., 2021), calculators (Cobbe et al., 2021; Thoppilan et al., 2022), translation systems (Thoppilan et…”
- PAL: Program-aided Language Models (2022, background): “…a wide range of reasoning tasks, including commonsense (Wei et al., 2021; Sanh et al., 2021; Madaan et al., 2022), mathematical (Lewkowycz et al., 2022; Wu et al., 2022; Mishra et al., 2022), and symbolic reasoning (Yao et al., 2022; Ahn et al., 2022), using few-shot prompting (Brown et al., 2020).”
- MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action (2023, methodology): “It is also shown to be beneficial in NLP studies [34] to incorporate such reasoning.”
- Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks (2022, methodology): “ReAct Yao et al. (2022) propose to leverage external tools like search engine to enhance the LLM reasoning skills.”
- Measuring and Narrowing the Compositionality Gap in Language Models (2022, methodology): “In parallel to our work Khot et al. (2023) and Yao et al. (2023) present methods similar to self-ask, but they do not present our findings on the compo-sitionality gap, integrate their prompt with a web search engine or present new datasets as in our Compositional Celebrities and Bamboogle.”
- Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity (2024, result): “In contrast to this decomposition-based approach, other recent studies, such as Yao et al. (2023) and Trivedi et al. (2023), explored the interleaving of Chain-of-Thought reasoning (Wei et al., 2022b) — a method where a logical sequence of thoughts is generated — with document retrieval, repeatedly…”
- How is ChatGPT's behavior changing over time? (2023, methodology): “Specifically, we used the default ReAct Agent in LangChain(3) (designed to reproduce ReAct prompting [YZY22]) with different LLMs (GPT-4 and GPT-3.”
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs (2023, neutral): “Alternatively, natu-ral language knowledge can be retrieved from exter-Ability nal sources, as demonstrated by WebGPT (Nakano et al., 2021) and ReAct (Yao et al., 2022) through the use of search APIs.”
- A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis (2023, background): “Large language models (LLM) [8, 11, 49, 50, 68] can solve variety of natural language tasks, such as arithmetic, commonsense, logical reasoning, question answering, text generation [8, 31, 77, 78], and even interactive decision making tasks [2, 82].”
- FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation (2023, neutral): “10 We retrieve all of the search results, including the answer box , organic results , and other useful information, such as the knowledge graph , questions and answers from crowdsourced QA platforms, and related questions that search users also ask (see Figure 6 in Appendix F).”
- ToolQA: A Dataset for LLM Question Answering with External Tools (2023, methodology): “(5) ReAct [76] integrates reasoning with tool use by prompting LLMs to generate interleaved verbal reasoning traces and tool calls.”
- A Survey on Large Language Models for Code Generation (2024, neutral): “CodeBLEU combines these elements (including weighted n-gram match, syntactic AST match, and semantic data-flow match) into a comprehensive evaluation metric.”
- ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (2024, background): “It is widely observed that LLMs are capable to serve as intelligent agents in versatile environments and contexts [30; 51], known as LLMs-as-Agents [25].”
- Enabling Large Language Models to Generate Text with Citations (2023, methodology): “In I NLI - NE S EARCH , we allow LLMs to call “search” during the generation process (Yao et al., 2023; Press et al., 2022; Jiang et al., 2023).”
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society (2023, methodology): “Special types of instruction or prompting methods , such as Chain-of-Thought (CoT) [123], zero-shot-CoT [61], and ReAct [126], have recently been developed to enhance the performance of LLMs on reasoning, arithmetic and decision making tasks [134, 118, 52, 73, 31, 103, 43, 64, 132, 46, 133, 105,…”
- Gorilla: Large Language Model Connected with Massive APIs (2023, neutral): “Researchers have proposed an array of strategies to prompt LLMs to perform better in coding tasks, including in-context learning [44, 18, 7], task decomposition [17, 46], and self-debugging [8, 34].”
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts (2023, methodology): “For such instances, we regain the relevant passage from Wikipedia, ensuring it includes the ground truth.”