MemoryBank: Enhancing Large Language Models with Long-Term Memory

By Wanjun Zhong, Lianghong Guo, Qi-Fei Gao, He Ye, Yanlin Wang. AAAI Conference on Artificial Intelligence, 2023

555 citations recorded by the source corpus.

Large Language Models (LLMs) have drastically reshaped our interactions with artificial intelligence (AI) systems, showcasing impressive performance across an extensive array of tasks. Despite this, a notable hindrance remains—the deficiency of a long-term memory mechanism within these models. This shortfall becomes increasingly evident in situations demanding sustained interaction, such as personal companion systems, psychological counseling, and secretarial assistance. Recognizing the necessity for long-term memory, we propose MemoryBank, a novel memory mechanism tailored for LLMs. MemoryBank enables the models to summon relevant memories, continually evolve through continuous memory updates, comprehend, and adapt to a user's personality over time by synthesizing information from previous interactions. To mimic anthropomorphic behaviors and selectively preserve memory, MemoryBank incorporates a memory updating mechanism, inspired by the Ebbinghaus Forgetting Curve theory. This mechanism permits the AI to forget and reinforce memory based on time elapsed and the relative significance of the memory, thereby offering a more human-like memory mechanism and enriched user experience. MemoryBank is versatile in accommodating both closed-source models like ChatGPT and open-source models such as ChatGLM. To validate MemoryBank's effectiveness, we exemplify its application through the creation of an LLM-based chatbot named SiliconFriend in a long-term AI Companion scenario. Further tuned with psychological dialog data, SiliconFriend displays heightened empathy and discernment in its interactions. Experiment involves both qualitative analysis with real-world user dialogs and quantitative analysis with simulated dialogs. In the latter, ChatGPT acts as multiple users with diverse characteristics and generates long-term dialog contexts covering a wide array of topics. The results of our analysis reveal that SiliconFriend, equipped with MemoryBank, exhibits a strong capability for long-term companionship as it can provide emphatic response, recall relevant memories and understand user personality.

Subfields: Artificial Intelligence, Health Informatics

View Paper: https://www.semanticscholar.org/paper/c3a59e1e405e7c28319e5a1c5b5241f9b340cf63

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kai Zhang
2. Yu Su
3. Defu Lian
4. Renze Lou
5. Jian Xie
6. Xingmei Wang
7. Xiaolong Chen
8. Enhong Chen

Citations in the corpus

• A Comprehensive Overview of Large Language Models (2023, unknown): “Aside from this, [264] shows tool documentation is enough to teach LLMs to use tools without demonstrations.”
• Empowering Biomedical Discovery with AI Agents (2024, unknown): “For instance, ChatDB [142] uses an external database for memory storage, and MemoryBank [151] encodes memory segments into embeddings for later retrieval.”
• TravelPlanner: A Benchmark for Real-World Planning with Language Agents (2024, methodology): “Techniques like memory summarization (Chen et al., 2023; Zhou et al., 2023; Liang et al., 2023) and retrieval (Andreas, 2022; Park et al., 2023; Zhong et al., 2023) are widely employed to enhance the memory capabilities of language agents.”
• Understanding the planning of LLM agents: A survey (2024, methodology): “…in additional storage, and the forms are diverse, such as texts [Park et al. , 2023; Liu et al. , 2023b; Packer et al. , 2023; Wang et al. , 2023c; Zhong et al. , 2023], 2023c] encode each memory using a text encoding model into a vector and establish an indexing structure, such as FAISS library…”
• A-MEM: Agentic Memory for LLM Agents (2025, unknown): “Existing memory systems [25, 39, 28, 21] for LLM agents provide basic memory storage functionality.”
• The rise and potential of large language model based agents: a survey (2025, unknown): “Users might inadvertently disclose sensitive personal information during their interactions, which will be retained in the agent’s memory for extended periods [137].”
• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (2024, unknown): “For instance, MemoryBank (Zhong et al., 2024) and PerLTQA (Du et al., 2024) insufficiently evaluate the ability to synthesize information across numerous sessions or to reason with temporal metadata or time references.”
• A Survey on the Memory Mechanism of Large Language Model-based Agents (2024, unknown): “In addition to accumulating experience through self-involving trials, absorbing external knowledge as part of the agent’s memory is also critical to enhance the exploratory capabilities of the agent.”
• A survey on large language model based autonomous agents (2023, unknown): “Model Profile Memory Planning Action LS Time Operation Structure WebGPT[105] - 2 2 12/2021 SayCan[2] 1 2 3 04/2022 MRKL[71] 1 2 - 05/2022 Inner Monologue[64] 2 2 3 07/2022 Social Simulacra[110] 2 - 1 - 08/2022 ReAct[151] 2 2 3 10/2022 REPLUG[124] 2 1 - 1 - 01/2023 MALLM[121] 2 2 - 1 - 01/2023 DEPS[1”
• eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents (2026, unknown): “MemoryBank [Zhong et al., 2024] applies the Ebbinghaus forgetting curve, while eMEM implements an analogous decay through tier promotion and archival.”
• Constrained Dominant Sets for Multimodal Document Question Answering (2026, unknown): “The candidate set is constructed following the approach of agentic memory systems, such as MemGPT [39], MemoryBank [40], and A-MEM [26], where each document element is transformed into a memory note enriched with summaries, keywords, tags, and explicit links.”