Trained Persistent Memory for Frozen Decoder-Only LLMs

• URL: http://arxiv.org/abs/2603.22329v1
• Date: Fri, 20 Mar 2026 19:11:02 GMT
• Title: Trained Persistent Memory for Frozen Decoder-Only LLMs
• Abstract summary: We show that trained memory adapters give a frozen encoder-decoder backbone persistent latent-space memory.<n>On LoCoMo we find a striking inductive-bias dichotomy: at $1times$ capacity, three methods with strong architectural priors achieve retained-memory scores of $7-18%$.<n>These findings establish persistent latent-space memory as a general paradigm spanning major transformer families.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decoder-only language models are stateless: hidden representations are discarded after every forward pass and nothing persists across sessions. Jeong (2026a) showed that trained memory adapters give a frozen encoder-decoder backbone persistent latent-space memory, building on the lateral-memory framework of Jeong (2026b,c). Here we ask whether the same principle transfers to the decoder-only setting, where no cross-attention pathway exists and memory must enter through self-attention alone. We adapt six methods -- prefix, parallel cross-attention, KV extension, Hebbian memory, context-gated branch, and slot-based sparse write -- to a frozen GPT-2, training only a small adapter $θ_{mem}$. The write rule is shared; only the read injection changes from decoder cross-attention to self-attention KV prefix or parallel branch. On LoCoMo we find a striking inductive-bias dichotomy: at $1\times$ capacity, three methods with strong architectural priors -- cross-attention (M.2), Hebbian (M.4), and slot write (M.6) -- achieve retained-memory scores of $7-18\%$ and knowledge gains $ΔK$ of $7-10$, while the other three fail ($< 0.4\%$). At $10\times$ capacity all six converge, showing the gap is architectural, not fundamental. Together with the encoder-decoder results of Jeong (2026a) and the brain-inspired modules of Jeong (2026b,c), these findings establish persistent latent-space memory as a general paradigm spanning major transformer families.

Related papers

• Where Should Experience Live? Hierarchical Hebbian Memory for Continual Vision Transformers [0.0]
  Vision Transformers provide strong visual representations but typically rely on slowly updated parameters.<n>This work proposes textitHierarchical Hebbian Memory, a three-level memory architecture composed of rapid Working Memory, persistent Routed Episodic Memory, and slower Semantic Memory.
  arXiv  Detail & Related papers  (2026-08-31T20:51:51Z)
• Maglev: Sliding Recurrent Memory [8.788753986099902]
  ours is a recurrent Transformer architecture with fixed-size memory that generalizes sliding-window attention.<n>We train ours with a memory consistency loss that aligns $m_t$ with $m'_t$, allowing inference to use $P$ alone.
  arXiv  Detail & Related papers  (2026-08-03T20:40:49Z)
• AutoMem: Automated Learning of Memory as a Cognitive Skill [63.07128029793901]
  We show that memory expertise is a learned skill.<n>We introduce AutoMem, a framework that automates the two axes of memory management.<n>Our results show that memory management is an independently learnable skill.
  arXiv  Detail & Related papers  (2026-07-01T17:57:03Z)
• Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory [65.39827296429527]
  Scaling conditional memory offers a promising way to increase language-model capacity.<n>Existing methods such as Engram learn large memory tables from scratch during pre-training.<n>We propose Memory Grafting, a conditional memory scaling method that utilizes frozen hidden states from a grafting model as conditional n-gram memory.
  arXiv  Detail & Related papers  (2026-05-20T09:35:04Z)
• CAMeR: Keyword-Gated Hybrid Activation for Adaptive Memory Retention in LLM Agents [0.7977229957867867]
  Large language model (LLM) agents operating over extended dialogues accumulate vast amounts of information.<n>We present CAMeR, a memory retention framework combining keyword-gated hybrid activation.<n>On CAMeR-Bench, CAMeR's keyword gate achieves a 1.6$times$ larger retention gap between high-frequency and never-referenced memories.
  arXiv  Detail & Related papers  (2026-05-15T01:38:30Z)
• ZenBrain: A Neuroscience-Inspired 7-Layer Memory Architecture for Autonomous AI Systems [51.56484100374058]
  On LongMemEval-500, ZenBrain matches a long-context oracle's binary-judge accuracy to within 4.5 pp.<n> ZenBrain is a 7-layer neuroscience-inspired memory architecture.
  arXiv  Detail & Related papers  (2026-04-26T20:39:19Z)
• SuperLocalMemory V3.3: The Living Brain -- Biologically-Inspired Forgetting, Cognitive Quantization, and Multi-Channel Retrieval for Zero-LLM Agent Memory Systems [0.0]
  We present SuperLocalMemory V3.3 ("The Living Brain"), a local-first agent memory system.<n>It implements the full cognitive memory taxonomy with mathematical lifecycle dynamics.<n>It is open source under the Elastic License 2.0, runs entirely on CPU, and has over 5,000 monthly downloads.
  arXiv  Detail & Related papers  (2026-04-06T08:27:26Z)
• MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolution [52.29303869050117]
  Memory-augmented LLM agents maintain external memory banks to support long-horizon interaction.<n>MemMA is a plug-and-play multi-agent framework that coordinates the memory cycle along both the forward and backward paths.
  arXiv  Detail & Related papers  (2026-03-19T10:15:59Z)
• Trained Persistent Memory for Frozen Encoder--Decoder LLMs: Six Architectural Methods [0.0]
  We show that persistent memory in a frozen language model is feasible -- even under severe resource constraints.<n>We implement six architectural methods spanning three injection points and four write mechanisms.<n>We argue that full end-to-end training with larger models, larger data, and orders-of-magnitude larger memory will yield substantially stronger results.
  arXiv  Detail & Related papers  (2026-03-17T11:51:21Z)
• KEEP: A KV-Cache-Centric Memory Management System for Efficient Embodied Planning [8.216400469571084]
  We propose KEEP, a KV-cache-centric memory management system for efficient embodied planning.<n>KEEP features 3 key innovations: (1) a Static-Dynamic Memory Construction algorithm that reduces KV cache recomputation by mixed-granularity memory group; (2) a Multi-hop Memory Re-computation algorithm that dynamically identifies important cross-attention among different memory groups; and (3) a Layer-balanced Memory Loading that eliminates unbalanced KV cache loading and cross-attention across different layers.
  arXiv  Detail & Related papers  (2026-02-27T01:48:07Z)
• TinyVLM: Zero-Shot Object Detection on Microcontrollers via Vision-Language Distillation with Matryoshka Embeddings [0.14504054468850666]
  We present TinyVLM, the first framework enabling zero-shot object detection on resource-constrained MCUs with less than 1MB of memory.<n>TinyVLM achieves competitive zero-shot accuracy on COCO, Flowers102, and Food101 while requiring only 285KB of RAM and 892KB of flash memory.
  arXiv  Detail & Related papers  (2026-02-24T05:32:51Z)
• Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution [52.76038908826961]
  We propose $textbfReMe$ ($textitRemember Me, Refine Me$) to bridge the gap between static storage and dynamic reasoning.<n>ReMe innovates across the memory lifecycle via three mechanisms: $textitmulti-faceted distillation$, which extracts fine-grained experiences.<n>Experiments on BFCL-V3 and AppWorld demonstrate that ReMe establishes a new state-of-the-art in agent memory system.
  arXiv  Detail & Related papers  (2025-12-11T14:40:01Z)
• Agentic Learner with Grow-and-Refine Multimodal Semantic Memory [50.81667005063605]
  ViLoMem is a dual-stream memory framework that constructs compact, schema-based memory.<n>It encodes visual distraction patterns and logical reasoning errors, enabling MLLMs to learn from their successful and failed experiences.
  arXiv  Detail & Related papers  (2025-11-26T18:55:08Z)
• MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation [59.31354761628506]
  Temporal context is essential for robotic manipulation because such tasks are inherently non-Markovian, yet mainstream VLA models typically overlook it.<n>We propose MemoryVLA, a Cognition-Memory-Action framework for long-horizon robotic manipulation.<n>We evaluate it on 150+ simulation and real-world tasks across three robots.
  arXiv  Detail & Related papers  (2025-08-26T17:57:16Z)
• G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems [44.844636264484905]
  Large language model (LLM)-powered multi-agent systems (MAS) have demonstrated cognitive and execution capabilities that far exceed those of single LLM agents.<n>We introduce G-Memory, a hierarchical, agentic memory system for MAS inspired by organizational memory theory.<n>G-Memory improves success rates in embodied action and accuracy in knowledge QA by up to $20.89%$ and $10.12%$, respectively.
  arXiv  Detail & Related papers  (2025-06-09T03:43:46Z)
• Efficient Beam Search for Large Language Models Using Trie-Based Decoding [13.496312069446963]
  This work presents a novel trie (-tree)-based parallel decoding method that addresses the memory inefficiency of batch-based beam search.<n>By sharing a single KV cache across beams with common prefixes, our approach dramatically reduces memory usage and enables efficient decoding.
  arXiv  Detail & Related papers  (2025-01-31T16:22:36Z)
• You Only Cache Once: Decoder-Decoder Architectures for Language Models [132.4064488592704]
  We introduce a decoder-decoder architecture, YOCO, for large language models. YOCO only caches key-value pairs once. The overall model behaves like a decoder-only Transformer, although YOCO only caches once.
  arXiv  Detail & Related papers  (2024-05-08T17:57:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.