Metis: Memory Foundation Model

• URL: http://arxiv.org/abs/2607.26760v2
• Date: Tue, 04 Aug 2026 08:51:55 GMT
• Title: Metis: Memory Foundation Model
• Abstract summary: We introduce memory foundation models, which empower foundation models with native memory capabilities.<n>We show that native memory offers advantages in architecture, end-to-end optimization, and efficiency.<n>We present Metis, the first prototype of memory foundation models.
• Score: 96.59067961902532
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in AI agents have increasingly internalized native capabilities into their underlying foundation models, giving rise to multimodal foundation models and large reasoning models. However, agent memory is still primarily implemented through external modules, leaving the native memory capability largely unexplored. In this paper, we take a first step toward this direction by introducing memory foundation models, which empower foundation models with native memory capabilities. We formalize native memory from two perspectives: a persistent and dynamically evolving memory state within the backbone, and native memory procedures that autonomously store and utilize information through model computation. We show that native memory offers advantages in architecture, end-to-end optimization, and efficiency. Based on this formulation, we propose Metis, the first prototype of memory foundation models. Metis introduces a new architecture that equips a foundation model with a native memory state, allowing historical information to be compressed into the model and accessed through memory attention. We construct large-scale memory-specific training data and introduce multiple optimization objectives to acquire these native memory procedures through mid-training. The online memory maintenance of Metis is gradient-free, and the memory update requires only a forward pass. At inference time, all learned model weights remain frozen, while the native memory states are autonomously transformed through standard forward computation. Through extensive experiments, we show that Metis exhibits native memory capabilities and further provide a detailed analysis of its strengths, limitations, and behaviors. To facilitate future research on memory foundation models, we release our project and model checkpoints.

Related papers

• AutoMem: Automated Learning of Memory as a Cognitive Skill [63.07128029793901]
  We show that memory expertise is a learned skill.<n>We introduce AutoMem, a framework that automates the two axes of memory management.<n>Our results show that memory management is an independently learnable skill.
  arXiv  Detail & Related papers  (2026-07-01T17:57:03Z)
• M$^3$Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks [19.25978075323521]
  M$3$Eval is the first comprehensive evaluation framework and benchmark for probing different memory dimensions in multi-modal models.<n>We conduct experiments across representative multi-modal models, revealing consistent weaknesses and distinctive behaviors.<n>Our findings highlight memory as a fundamental yet underexplored capability and offer insights for designing more effective memory mechanisms in multi-modal models.
  arXiv  Detail & Related papers  (2026-06-03T15:28:57Z)
• MemEvolve: Meta-Evolution of Agent Memory Systems [66.09735157017558]
  Self-evolving memory systems are unprecedentedly reshaping the evolutionary paradigm of large language model (LLM)-based agents.<n>MemeEvolve is a meta-evolutionary framework that jointly evolves agents' experiential knowledge and their memory architecture.<n> EvolveLab is a unified self-evolving memory that distills twelve representative memory systems into a modular design space.
  arXiv  Detail & Related papers  (2025-12-21T14:26:14Z)
• Memory in the Age of AI Agents [217.9368190980982]
  This work aims to provide an up-to-date landscape of current agent memory research.<n>We identify three dominant realizations of agent memory, namely token-level, parametric, and latent memory.<n>To support practical development, we compile a comprehensive summary of memory benchmarks and open-source frameworks.
  arXiv  Detail & Related papers  (2025-12-15T17:22:34Z)
• MemoryKT: An Integrative Memory-and-Forgetting Method for Knowledge Tracing [7.096160553754792]
  Simulating students' memory states is a promising approach to enhance both the performance and interpretability of knowledge tracing models.<n>Memory consists of three fundamental processes: encoding, storage, and retrieval.<n>This paper proposes memoryKT, a knowledge tracing model based on a novel temporal variational autoencoder.
  arXiv  Detail & Related papers  (2025-08-11T15:59:59Z)
• Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning [64.93848182403116]
  Current deep-learning memory models struggle in reinforcement learning environments that are partially observable and long-term. We introduce the Stable Hadamard Memory, a novel memory model for reinforcement learning agents. Our approach significantly outperforms state-of-the-art memory-based methods on challenging partially observable benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T03:50:17Z)
• MEMORYLLM: Towards Self-Updatable Large Language Models [101.3777486749529]
  Existing Large Language Models (LLMs) usually remain static after deployment. We introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool. MEMORYLLM can self-update with text knowledge and memorize the knowledge injected earlier.
  arXiv  Detail & Related papers  (2024-02-07T07:14:11Z)
• Training Language Models with Memory Augmentation [28.4608705738799]
  We present a novel training approach designed for training language models with memory augmentation. Our approach uses a training objective that directly takes in-batch examples as accessible memory. We demonstrate significant gains over previous memory-augmented approaches.
  arXiv  Detail & Related papers  (2022-05-25T11:37:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.