Cognitive Memory in Large Language Models

• URL: http://arxiv.org/abs/2504.02441v2
• Date: Thu, 24 Apr 2025 01:47:25 GMT
• Title: Cognitive Memory in Large Language Models
• Authors: Lianlei Shan, Shixian Luo, Zezhou Zhu, Yu Yuan, Yong Wu,
• Abstract summary: This paper examines memory mechanisms in Large Language Models (LLMs), emphasizing their importance for context-rich responses, reduced hallucinations, and improved efficiency.<n>It categorizes memory into sensory, short-term, and long-term, with sensory memory corresponding to input prompts, short-term memory processing immediate context, and long-term memory implemented via external databases or structures.
• Score: 8.059261857307881
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper examines memory mechanisms in Large Language Models (LLMs), emphasizing their importance for context-rich responses, reduced hallucinations, and improved efficiency. It categorizes memory into sensory, short-term, and long-term, with sensory memory corresponding to input prompts, short-term memory processing immediate context, and long-term memory implemented via external databases or structures. The text-based memory section covers acquisition (selection and summarization), management (updating, accessing, storing, and resolving conflicts), and utilization (full-text search, SQL queries, semantic search). The KV cache-based memory section discusses selection methods (regularity-based summarization, score-based approaches, special token embeddings) and compression techniques (low-rank compression, KV merging, multimodal compression), along with management strategies like offloading and shared attention mechanisms. Parameter-based memory methods (LoRA, TTT, MoE) transform memories into model parameters to enhance efficiency, while hidden-state-based memory approaches (chunk mechanisms, recurrent transformers, Mamba model) improve long-text processing by combining RNN hidden states with current methods. Overall, the paper offers a comprehensive analysis of LLM memory mechanisms, highlighting their significance and future research directions.

Related papers

• Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions [55.19217798774033]
  Memory is a fundamental component of AI systems, underpinning large language models (LLMs) based agents. We introduce six fundamental memory operations: Consolidation, Updating, Indexing, Forgetting, Retrieval, and Compression. This survey provides a structured and dynamic perspective on research, benchmark datasets, and tools related to memory in AI.
  arXiv  Detail & Related papers  (2025-05-01T17:31:33Z)
• Quantifying Memory Utilization with Effective State-Size [73.52115209375343]
  We develop a measure of textitmemory utilization' This metric is tailored to the fundamental class of systems with textitinput-invariant and textitinput-varying linear operators
  arXiv  Detail & Related papers  (2025-04-28T08:12:30Z)
• R$^3$Mem: Bridging Memory Retention and Retrieval via Reversible Compression [24.825945729508682]
  We propose R$3$Mem, a memory network that optimize both information Retention and Retrieval.<n>R$3$Mem employs virtual memory tokens to compress and encode infinitely long histories, further enhanced by a hierarchical compression strategy.<n>Experiments demonstrate that our memory design achieves state-of-the-art performance in long-context language modeling and retrieval-augmented generation tasks.
  arXiv  Detail & Related papers  (2025-02-21T21:39:00Z)
• On Memory Construction and Retrieval for Personalized Conversational Agents [69.46887405020186]
  We propose SeCom, a method that constructs the memory bank at segment level by introducing a conversation segmentation model.<n> Experimental results show that SeCom exhibits a significant performance advantage over baselines on long-term conversation benchmarks LOCOMO and Long-MT-Bench+.
  arXiv  Detail & Related papers  (2025-02-08T14:28:36Z)
• Memory-Driven Metaheuristics: Improving Optimization Performance [0.0]
  This chapter explores the significance of memory in metaheuristic algorithms.<n>The key factors influencing the effectiveness of memory mechanisms are discussed.<n>A comprehensive analysis of how memory mechanisms are incorporated into popular metaheuristic algorithms is presented.
  arXiv  Detail & Related papers  (2024-11-07T13:27:03Z)
• Empowering Working Memory for Large Language Model Agents [9.83467478231344]
  This paper explores the potential of applying cognitive psychology's working memory frameworks to large language models (LLMs) An innovative model is proposed incorporating a centralized Working Memory Hub and Episodic Buffer access to retain memories across episodes. This architecture aims to provide greater continuity for nuanced contextual reasoning during intricate tasks and collaborative scenarios.
  arXiv  Detail & Related papers  (2023-12-22T05:59:00Z)
• SCM: Enhancing Large Language Model with Self-Controlled Memory Framework [54.33686574304374]
  Large Language Models (LLMs) are constrained by their inability to process lengthy inputs, resulting in the loss of critical historical information.<n>We propose the Self-Controlled Memory (SCM) framework to enhance the ability of LLMs to maintain long-term memory and recall relevant information.
  arXiv  Detail & Related papers  (2023-04-26T07:25:31Z)
• LaMemo: Language Modeling with Look-Ahead Memory [50.6248714811912]
  We propose Look-Ahead Memory (LaMemo) that enhances the recurrence memory by incrementally attending to the right-side tokens. LaMemo embraces bi-directional attention and segment recurrence with an additional overhead only linearly proportional to the memory length. Experiments on widely used language modeling benchmarks demonstrate its superiority over the baselines equipped with different types of memory.
  arXiv  Detail & Related papers  (2022-04-15T06:11:25Z)
• Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory [75.65949969000596]
  Episodic and semantic memory are critical components of the human memory model. We develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory. We demonstrate that this allocation scheme improves performance in memory conditional image generation.
  arXiv  Detail & Related papers  (2021-02-20T18:40:40Z)
• Distributed Associative Memory Network with Memory Refreshing Loss [5.5792083698526405]
  We introduce a novel Distributed Associative Memory architecture (DAM) with Memory Refreshing Loss (MRL) Inspired by how the human brain works, our framework encodes data with distributed representation across multiple memory blocks. MRL enables MANN to reinforce an association between input data and task objective by reproducing input data from stored memory contents.
  arXiv  Detail & Related papers  (2020-07-21T07:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.