Reflexion: an autonomous agent with dynamic memory and self-reflection

By Noah Shinn, Beck Labash, A. Gopinath. 2023

338 citations recorded by the source corpus.

Subfields: Prompting & Reasoning, LLM Agents & Tool Use

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Prompting & Reasoning

In LLM Agents & Tool Use

Citations in the corpus

Self-Refine: Iterative Refinement with Self-Feedback (2023, methodology): “Compared to a concurrent work, Reflexion (Shinn et al., 2023), our approach involves correction using feedback, whereas their setup involves finding the next best solution in planning using ReAct.”
Large Language Models as Optimizers (2023, positive): “…the model output, which has shown effectiveness in reducing harmful LLM outputs (Bai et al., 2022; Ganguli et al., 2023), improving reasoning (Shinn et al., 2023; Madaan et al., 2023) and code generation performance (Chen et al., 2023e; Olausson et al., 2023; Shinn et al., 2023; Chen et al.,…”
Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning (2023, background): “Other works have aimed to improve the planning performance of LLMs through prompt engineering [60] or collecting various forms of feedback such as sensory information [51, 20, 34], human corrections [60], self-corrections [46] or information of unmet preconditions [42, 56].”
Training Language Models to Self-Correct via Reinforcement Learning (2024, neutral): “…in the ﬁrst stage, it produces an initialization that is less susceptible to behavior collapse by training to correct second-attempt responses while constraining the ﬁrst-turn distribution to be close to the base model; followed by training on both attempts to maximize reward in the second stage.”
Teaching Large Language Models to Self-Debug (2023, background): “, 2023) and improves the performance on some reasoning tasks (Shinn et al., 2023; Madaan et al., 2023b; Kim et al., 2023; Nair et al., 2023).”
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases (2024, positive): “LLM Agent based on RAG LLM Agents have demonstrated powerful reasoning and interaction capability in many real-world settings, spanning from autonomous driving [22, 38, 6], knowledge-intensive question-answering [36, 26, 16], and healthcare [25, 1].”
WebArena: A Realistic Web Environment for Building Autonomous Agents (2023, background): “Moreover, it is crucial for LLMs to include built-in failure recovery and self-correction mechanisms to increase their robustness [28, 14].”
AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents (2024, background): “It is zero-shot, integrating ReAct-style [65] and Reflexion-style [46] prompting to consume user instructions and screen content, reason, take actions, and update its decision-making based on the outcome of its actions.”
Decomposition Enhances Reasoning via Self-Evaluation Guided Decoding (2023, positive): “Inspired by the recent success of self-evaluation (Kadavath et al., 2022; Shinn et al., 2023; Madaan et al., 2023; Paul et al., 2023), a scheme to prompt LLMs to evaluate their own generation, we use LLMs to judge the faithfulness of st based on s1:t−1.”
Large Language Model Guided Tree-of-Thought (2023, background): “Building on top of ReAct, Reflexion [27] is an approach that equips an agent with dynamic memory and self-reflection capabilities, improving its existing reasoning trace and ability to choose task-specific actions.”
Voyager: An Open-Ended Embodied Agent with Large Language Models (2023, methodology): “The items collected by Reflexion [30] in each trial is”
AgentBench: Evaluating LLMs as Agents (2023, methodology): “Despite many improved strategies proposed later, such as introducing ensemble [87], reflection [70], and search [98], we evaluate LLMs with the most primitive CoT in AgentBench, as it reflects the most practical public user experience without multiple trails and repeated generation.”
MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (2023, methodology): “Note that when evaluating on sparse feedback setting, MINT is equivalent to the setting of Reflexion feedback (Shinn et al., 2023).”
Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training (2023, neutral): “Besides, a more relevant series of work has focused on enhancing reasoning through evaluations, including learned reward models (Uesato et al., 2022; Lightman et al., 2023) and self-evaluation (Shinn et al., 2023; Madaan et al., 2023).”
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf (2023, background): “Shinn et al. (2023) and Fu et al. (2023) are the most similar works to ours.”
MetaGPT: Meta Programming for Multi-Agent Collaborative Framework (2023, methodology): “ReAct [31] utilizes Chain of Thought prompts [34] to generate reasoning trajectories and action plans with LLMs. Reflexion [32] infers more intuitive future actions through self-reflection.”
Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions (2023, background): “However, adapting LLMs for decision-making tasks often involves non-trivial prompt design, memory retrieval mechanisms to dynamically construct the agent’s context [12, 17, 25], and sometimes model fine-tuning [9] to enhance its decision-making abilities.”
Gorilla: Large Language Model Connected with Massive APIs (2023, positive): “Researchers have proposed an array of strategies to prompt LLMs to perform better in coding tasks, including in-context learning [44, 18, 7], task decomposition [17, 46], and self-debugging [8, 34].”