ExpeL: LLM Agents Are Experiential Learners

By Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Y. Liu, Gao Huang. AAAI Conference on Artificial Intelligence, 2023

699 citations recorded by the source corpus.

The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs. While there is a growing demand to tailor LLMs for custom decision-making tasks, finetuning them for specific tasks is resource-intensive and may diminish the model's generalization capabilities. Moreover, state-of-the-art language models like GPT-4 and Claude are primarily accessible through API calls, with their parametric weights remaining proprietary and unavailable to the public. This scenario emphasizes the growing need for new methodologies that allow learning from agent experiences without requiring parametric updates. To address these problems, we introduce the Experiential Learning (ExpeL) agent. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions. Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences. We further explore the emerging capabilities and transfer learning potential of the ExpeL agent through qualitative observations and additional experiments.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/5e4597eb21a393b23e473cf66cb5ae8b27cab03e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The rise and potential of large language model based agents: a survey (2025, neutral): “Using prompts, some methods succinctly integrate memories [135], while others emphasize reﬂective processes to create condensed memory representations [16,136].”
DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (2023, methodology): “…using model selection techniques like cross-validation or, in principle, with sophisticated techniques involving RL and LM feedback (Hu et al., 2023; Zhao et al., 2023a; Shinn et al., 2023) or learned or Bayesian hyperparameter optimization methods (Bergstra et al., 2013; Akiba et al., 2019).”
ChatDev: Communicative Agents for Software Development (2023, background): “…use (Schick et al., 2023; Cai et al., 2023; Qin et al., 2023a; Ruan et al., 2023; Yang et al., 2023b), enabling them to reason in complex scenarios (Wei et al., 2022b; Zhao et al., 2023; Zhou et al., 2023a; Ma et al., 2023; Zhang et al., 2023a; Wang et al., 2023b; Ding et al., 2023; Weng, 2023).”
A survey on large language model based autonomous agents (2023, neutral): “In the following, we turn our focus to memory operations, which are used to interact with external environments.”
A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, neutral): “In the following parts, we present several representative tasks that are leveraged to evaluate the memory module in indirect ways.”
Traj-Evolve: A Self-Evolving Multi-Agent System for Patient Trajectory Modeling in Lung Cancer Early Detection (2026, unknown): “For example, memory-evolving methods such as Reflexion (Shinn et al., 2023), ExpeL (Zhao et al., 2024), Memento (Zhou et al., 2025a), and Agent KB (Tang et al., 2025) store and retrieve past trajectories as non-parametric experience.”
STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task Planning (2025, unknown): “The construction of embodied agents can be categorized into three approaches, including rule-based and traditional planning methods [13], RL methods [14], and LLM-based methods [15].”