MemoryWAM: Efficient World Action Modeling with Persistent Memory
- URL: http://arxiv.org/abs/2606.20562v1
- Date: Thu, 18 Jun 2026 17:59:51 GMT
- Title: MemoryWAM: Efficient World Action Modeling with Persistent Memory
- Abstract summary: We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
- Score: 80.90899269062128
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Robust robotic manipulation in the real world requires not only an understanding of the current observation, but also memory and dynamics modeling. World action models (WAMs) possess these capabilities by jointly modeling visual foresight and actions conditioned on both current and historical observations, making them a promising paradigm for robotic manipulation. However, existing WAMs face a fundamental trade-off: methods with efficient inference typically condition only on a bounded window of recent observations and therefore struggle in non-Markovian environments, whereas methods that preserve long histories incur time and space costs that grow substantially with sequence length. To address this challenge, we introduce MemoryWAM, a world action model with efficient persistent memory. MemoryWAM uses a hybrid memory design that combines recent frames, event-boundary anchor frames, and compact gist tokens that summarize long-range history. A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context, supporting memory-dependent decision-making with reduced inference latency and GPU memory usage. Across long-horizon, memory-dependent manipulation tasks in both simulation and the real world, MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines while maintaining favorable computational efficiency.
Related papers
- HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation [12.167895321305464]
We present HiMem-WAM, a Hierarchical Memory-Gated WAM that integrates motion-centric latent actions, high-level skill latents, and boundary-triggered memory updates.<n>Specifically, we develop a hierarchical latent action framework that jointly learns low-level motion and high-level skill latents.<n>A boundary-aware memory gate writes compact task states at predicted skill transitions, enabling causal inference without test-time generation of future video or optical flow estimation.
arXiv Detail & Related papers (2026-06-09T03:22:34Z) - MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models [80.70528162709276]
Temporal modeling is essential for robotic manipulation, as effective control requires memory of past interactions and imagination of future states.<n>We propose MemoryVLA++, a full temporal modeling framework that equips VLA models with memory and imagination for robotic manipulation.<n>Our method achieves strong performance across Libero, SimplerEnv, Mikasa-Robo, Calvin, Libero-Plus, and diverse real-robot tasks.
arXiv Detail & Related papers (2026-06-08T17:59:53Z) - MemGround: Long-Term Memory Evaluation Kit for Large Language Models in Gamified Scenarios [33.8882826707344]
MemGround is a rigorous long-term memory benchmark grounded in rich, gamified interactive scenarios.<n>To comprehensively quantify both memory utilization and behavioral trajectories, we propose a multi-dimensional metric suite.
arXiv Detail & Related papers (2026-03-23T02:57:39Z) - RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies [54.23445842621374]
Memory is critical for long-horizon and history-dependent robotic manipulation.<n>Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms.<n>We introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models.
arXiv Detail & Related papers (2026-03-04T21:59:32Z) - RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
arXiv Detail & Related papers (2025-12-03T18:29:20Z) - FindingDory: A Benchmark to Evaluate Memory in Embodied Agents [49.18498389833308]
We introduce a new benchmark for long-range embodied tasks in the Habitat simulator.<n>This benchmark evaluates memory-based capabilities across 60 tasks requiring sustained engagement and contextual awareness.
arXiv Detail & Related papers (2025-06-18T17:06:28Z) - Long-Context State-Space Video World Models [66.28743632951218]
We propose a novel architecture leveraging state-space models (SSMs) to extend temporal memory without compromising computational efficiency.<n>Central to our design is a block-wise SSM scanning scheme, which strategically trades off spatial consistency for extended temporal memory.<n>Experiments on Memory Maze and Minecraft datasets demonstrate that our approach surpasses baselines in preserving long-range memory.
arXiv Detail & Related papers (2025-05-26T16:12:41Z) - WORLDMEM: Long-term Consistent World Simulation with Memory [20.450750381415965]
WorldMem is a framework that enhances scene generation with a memory bank consisting of memory units that store memory frames and states.<n>Our method is capable of accurately reconstructing previously observed scenes, even under significant viewpoint or temporal gaps.
arXiv Detail & Related papers (2025-04-16T17:59:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.