Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation

• URL: http://arxiv.org/abs/2607.07608v1
• Date: Wed, 08 Jul 2026 16:26:06 GMT
• Title: Dual Latent Memory in Vision-Language-Action Models for Robotic Manipulation
• Abstract summary: We introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens.<n>LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context.
• Score: 96.18832998704329
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mainstream Vision-Language-Action (VLA) models predict actions primarily from the current observation under a Markovian assumption, thus struggling with long-horizon, temporally dependent tasks. Existing memory-augmented VLAs either expand the observation window or retrieve history from the memory bank as auxiliary policy-side context. However, they leave memory outside the native latent embedding space of VLA reasoning, preventing historical experience from being fluidly interleaved with multimodal reasoning and action formation. To this end, we introduce LaMem-VLA, a latent-memory-native framework that reconstructs historical experience into latent memory tokens and directly interweaves them with VLA reasoning. At its core, LaMem-VLA introduces four coordinated components: (i) a curator that organizes historical experience into two complementary short-term and long-term memory vaults; (ii) a seeker that queries both vaults using the multimodal cognition to retrieve context-relevant evidence; (iii) a condenser that reconstructs the retrieved evidence into compact short-term and long-term latent memory tokens; and (iv) a weaver that injects these memory tokens with the current observation and instruction into one continuous embedding sequence. By representing, retrieving, and consuming historical experience entirely in the same continuous latent space, LaMem-VLA enables memory to directly participate in VLA reasoning and guide action generation under a bounded context. Extensive experiments on SimplerEnv and LIBERO demonstrate the superiority of our LaMem-VLA.

Related papers

• Remember Smarter: Visual History Compressor and Hyperbolic Experience Space for Robotic Memory [54.400138177707504]
  Remember Smarter (RS) is a plug-and-play module with complementary visual-history and hyperbolic experience-memory branches.<n>When adapted to pi0, RS increases total success on LIBERO-Plus from 53.6% to 70.6% and achieves substantial performance gains in real-robot experiments.
  arXiv  Detail & Related papers  (2026-08-15T15:13:33Z)
• MemoryWAM: Efficient World Action Modeling with Persistent Memory [80.90899269062128]
  We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
  arXiv  Detail & Related papers  (2026-06-18T17:59:51Z)
• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios [33.8882826707344]
  MemGround is a rigorous long-term memory benchmark grounded in rich, gamified interactive scenarios.<n>To comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite.
  arXiv  Detail & Related papers  (2026-03-23T02:57:39Z)
• ReMem-VLA: Empowering Vision-Language-Action Model with Memory via Dual-Level Recurrent Queries [45.23935281952228]
  We introduce ReMem-VLA, a vision-language-action (VLA) model equipped with two sets of learnable queries.<n>These queries are trained end-to-end to aggregate and maintain relevant context over time.<n>We demonstrate that ReMem-VLA exhibits strong memory capabilities across multiple dimensions.
  arXiv  Detail & Related papers  (2026-03-13T12:38:42Z)
• Notes-to-Self: Scratchpad Augmented VLAs for Memory Dependent Manipulation Tasks [9.55115186979077]
  We show a way to impart both spatial and temporal memory to a vision-language-action (VLA) model by incorporating a language scratchpad.<n>We evaluate this approach on a split of memory-dependent tasks from the ClevrSkills environment, on MemoryBench, and on a challenging real-world pick-and-place task.
  arXiv  Detail & Related papers  (2026-02-24T15:30:55Z)
• VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models [78.88575188716378]
  VisMem is a framework that equips Vision-Language Models with dynamic latent vision memories, a short-term module for fine-grained perceptual retention and a long-term module for abstract semantic consolidation.<n>Our experiments reveal that VisMem delivers a significant average performance boost of 11.8% relative to the vanilla model.
  arXiv  Detail & Related papers  (2025-11-14T06:51:34Z)
• MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation [59.31354761628506]
  Temporal context is essential for robotic manipulation because such tasks are inherently non-Markovian, yet mainstream VLA models typically overlook it.<n>We propose MemoryVLA, a Cognition-Memory-Action framework for long-horizon robotic manipulation.<n>We evaluate it on 150+ simulation and real-world tasks across three robots.
  arXiv  Detail & Related papers  (2025-08-26T17:57:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.