HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation

• URL: http://arxiv.org/abs/2606.10363v1
• Date: Tue, 09 Jun 2026 03:22:34 GMT
• Title: HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
• Abstract summary: We present HiMem-WAM, a Hierarchical Memory-Gated WAM that integrates motion-centric latent actions, high-level skill latents, and boundary-triggered memory updates.<n>Specifically, we develop a hierarchical latent action framework that jointly learns low-level motion and high-level skill latents.<n>A boundary-aware memory gate writes compact task states at predicted skill transitions, enabling causal inference without test-time generation of future video or optical flow estimation.
• Score: 12.167895321305464
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World Action Models (WAMs) have emerged as a new powerful paradigm for embodied intelligence, learning action-relevant visual dynamics that significantly enhance generalization and robustness. However, existing WAMs still struggle with task-relevant memory in long-horizon robotic manipulation. To address this, we present HiMem-WAM, a Hierarchical Memory-Gated WAM that integrates motion-centric latent actions, high-level skill latents, and boundary-triggered memory updates. Specifically, we develop a hierarchical latent action framework that jointly learns low-level motion and high-level skill latents, providing structured temporal abstraction. Meanwhile, a boundary-aware memory gate writes compact task states at predicted skill transitions, enabling causal inference without test-time generation of future video or optical flow estimation. Evaluated on LIBERO, LIBERO-PLUS, RMBench and real-world tasks, HiMem-WAM shows that hierarchical latents improve robustness under deployment perturbations, and the memory module substantially benefits memory-dependent long-horizon manipulation.

Related papers

• MemoryWAM: Efficient World Action Modeling with Persistent Memory [80.90899269062128]
  We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
  arXiv  Detail & Related papers  (2026-06-18T17:59:51Z)
• MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models [80.70528162709276]
  Temporal modeling is essential for robotic manipulation, as effective control requires memory of past interactions and imagination of future states.<n>We propose MemoryVLA++, a full temporal modeling framework that equips VLA models with memory and imagination for robotic manipulation.<n>Our method achieves strong performance across Libero, SimplerEnv, Mikasa-Robo, Calvin, Libero-Plus, and diverse real-robot tasks.
  arXiv  Detail & Related papers  (2026-06-08T17:59:53Z)
• BrainMem: Brain-Inspired Evolving Memory for Embodied Agent Task Planning [17.072340201392723]
  Embodied task planning requires agents to execute long-horizon, goal-directed actions in complex 3D environments.<n>We propose BrainMem, a training-free hierarchical memory system that equips embodied agents with working, episodic, and semantic memory inspired by human cognition.
  arXiv  Detail & Related papers  (2026-03-12T17:54:42Z)
• LatentMem: Customizing Latent Memory for Multi-Agent Systems [44.59989123744384]
  We propose LatentMem, a learnable multi-agent memory framework designed to customize agent-specific memories in a token-efficient manner.<n>Specifically, LatentMem comprises an experience bank that stores raw interaction trajectories in a lightweight form, and a memory composer that synthesizes compact latent memories conditioned on retrieved experience and agent-specific contexts.
  arXiv  Detail & Related papers  (2026-02-03T03:03:16Z)
• MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulation [59.31354761628506]
  Temporal context is essential for robotic manipulation because such tasks are inherently non-Markovian, yet mainstream VLA models typically overlook it.<n>We propose MemoryVLA, a Cognition-Memory-Action framework for long-horizon robotic manipulation.<n>We evaluate it on 150+ simulation and real-world tasks across three robots.
  arXiv  Detail & Related papers  (2025-08-26T17:57:16Z)
• RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems [41.89907261427986]
  Embodied agents face persistent challenges in real-world environments, including partial observability, limited spatial reasoning, and high-latency multi-memory integration.<n>We present RoboMemory, a brain-inspired framework that unifies Spatial, Temporal, Episodic, and Semantic memory under a parallelized architecture for efficient long-horizon planning and interactive environmental learning.<n> Experiments on EmbodiedBench show that RoboMemory, built on Qwen2.5-VL-72B-Ins, improves average success rates by 25% over its baseline and exceeds the closed-source state-of-the-art (SOTA) Gemini-1.5-
  arXiv  Detail & Related papers  (2025-08-02T15:39:42Z)
• FindingDory: A Benchmark to Evaluate Memory in Embodied Agents [49.18498389833308]
  We introduce a new benchmark for long-range embodied tasks in the Habitat simulator.<n>This benchmark evaluates memory-based capabilities across 60 tasks requiring sustained engagement and contextual awareness.
  arXiv  Detail & Related papers  (2025-06-18T17:06:28Z)
• Long-Context State-Space Video World Models [66.28743632951218]
  We propose a novel architecture leveraging state-space models (SSMs) to extend temporal memory without compromising computational efficiency.<n>Central to our design is a block-wise SSM scanning scheme, which strategically trades off spatial consistency for extended temporal memory.<n>Experiments on Memory Maze and Minecraft datasets demonstrate that our approach surpasses baselines in preserving long-range memory.
  arXiv  Detail & Related papers  (2025-05-26T16:12:41Z)
• From RAG to Memory: Non-Parametric Continual Learning for Large Language Models [6.380729797938521]
  retrieval-augmented generation (RAG) has become the dominant way to introduce new information.<n>Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some gaps, namely sense-making and associativity.<n>We propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks.
  arXiv  Detail & Related papers  (2025-02-20T18:26:02Z)
• Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning [64.93848182403116]
  Current deep-learning memory models struggle in reinforcement learning environments that are partially observable and long-term. We introduce the Stable Hadamard Memory, a novel memory model for reinforcement learning agents. Our approach significantly outperforms state-of-the-art memory-based methods on challenging partially observable benchmarks.
  arXiv  Detail & Related papers  (2024-10-14T03:50:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.