DIM-WAM: World-Action Modeling with Diverse Historical Event Memory

• URL: http://arxiv.org/abs/2606.27677v1
• Date: Fri, 26 Jun 2026 03:17:39 GMT
• Title: DIM-WAM: World-Action Modeling with Diverse Historical Event Memory
• Abstract summary: World-action models have shown promising robot-manipulation performance by jointly predicting future visual states and actions.<n>We introduce DiM-WAM, a memory-augmented world-action model that integrates multi-scale historical context, local future dynamics, and global task progress.<n>On RMBench, DiM-WAM raises average success from 28.4% with LingBot-VA to 69.8%, exceeding the explicit-memory Mem-0 baseline at 42.0%.
• Score: 23.350949759638056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World-action models have shown promising robot-manipulation performance by jointly predicting future visual states and actions. However, existing methods mainly rely on short-term history and short-horizon future prediction, which is insufficient for long-horizon tasks whose correct execution depends on earlier observations and task progress. Such temporally dependent tasks require effective use of complementary temporal information, including recent local context, cross-stage historical events, immediate future dynamics, and global task progress. To address long-term forgetting and poor awareness of the global task state, we introduce DiM-WAM, a memory-augmented world-action model that integrates multi-scale historical context, local future dynamics, and global task progress. The memory extracts compact visual event information from real observations, updates multiple memory banks through independent similarity-based merging, and then reads the bank-identity- and time-embedded long-term context to condition video and action denoising. A progress-supervision objective further encourages memory tokens to encode not only completed historical events but also the current task stage and its implications for the remaining task. On RMBench, DiM-WAM raises average success from 28.4% with LingBot-VA to 69.8%, exceeding the explicit-memory Mem-0 baseline at 42.0%. On four real-world Franka tasks, it improves average stage success from 70.7% to 91.5% and full-task success from 52.5% to 80.0%. Project page: https://wangkai-casia.github.io/dim-wam/{\texttt{https://wangkai-casia.github.io/dim-wam/}}.

Related papers

• Memory as Plans: World-Action Modeling with Memory-Grounded Planning [33.29460541345171]
  MaP-WAM is a Memory-as-Plans framework that decomposes memory-dependent world-action modeling into memory-grounded planning and plan-conditioned execution.<n>MaP-WAM achieves state-of-the-art performance on RMBench with an 83.3% success rate and attains 78.0% success on real-robot tasks.
  arXiv  Detail & Related papers  (2026-09-10T13:52:51Z)
• Latent Action as Intention Enables Efficient Future Imagination for World Action Models [20.031922437200524]
  We introduce **LAWA**, a WAM architecture that uses compact latent actions as an operational representation of future intentions.<n>On RoboCasa, LAWA achieves state-of-the-art average success rates of 65.6% and 80.8% in the few-shot and full data settings.<n>It also preserves the performance level of the matched Joint-WAM variant while requiring 42.9% lower inference latency.
  arXiv  Detail & Related papers  (2026-08-25T17:59:03Z)
• Recursive Experiential-Working Memory Evolution for Long-Horizon Agent Harnesses [83.32611586724165]
  Recuris is a Experiential-Working Memory architecture for long-horizon agent harnesses.<n>Working Memory tracks task progress and guides skill selection from Experiential Memory.<n>Across four long-horizon benchmarks and ten models, Recuris improves task success in 35 of the 37 completed model-benchmark pairs.
  arXiv  Detail & Related papers  (2026-08-25T17:56:35Z)
• Mimir: A Neuro-Symbolic Memory System with Dynamic Grounding for Embodied Agents in Interactive Environments [15.524473346555395]
  Long-horizon embodied task requires agents to act under partial observability while preserving both scene belief and execution progress.<n>We introduce Mimir, a neuro-symbolic memory that separates world memory from task memory and dynamically grounds them before each action.<n>Across tested backbones, Mimir consistently improves on different EB-ALFRED and EB-Habitat tasks, with maximum gains of 42.5% and average gains of 23.0%, respectively.
  arXiv  Detail & Related papers  (2026-08-05T14:57:24Z)
• WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory [50.19449886428936]
  We introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory.<n>The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning.<n>Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following, and in-context adaptation.
  arXiv  Detail & Related papers  (2026-07-21T08:25:37Z)
• KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policies [14.253600207124194]
  Memory can address this ambiguity by enabling policies to infer task progress from execution history.<n>We propose KEMO, a lightweight plug-in memory framework that automatically preserves kinematics associated with task-relevant state changes for VLA policies.<n>We evaluate KEMO on various real-world dualarm manipulation tasks spanning 2 to 6 scored subtasks, and trajectory length ranging from 830 steps to 2846 execution steps (durations from 28 to 95 seconds)
  arXiv  Detail & Related papers  (2026-06-22T16:57:43Z)
• MemoryWAM: Efficient World Action Modeling with Persistent Memory [80.90899269062128]
  We introduce MemoryWAM, a world action model with efficient persistent memory.<n>A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context.<n>MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines in both simulation and the real world.
  arXiv  Detail & Related papers  (2026-06-18T17:59:51Z)
• Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation [55.42006264038458]
  Action-conditioned world models have emerged as a promising paradigm for robot learning.<n>Mem-World is a memory-augmented multi-view action-conditioned world model.<n>We present W-VMem, a 4D wrist-view-centered surfel-indexed memory that anchors historical observations to temporally evolving surface elements.
  arXiv  Detail & Related papers  (2026-06-17T11:42:00Z)
• VLA-Pro: Cross-Task Procedural Memory Transfer for Vision-Language-Action Models [73.99344788183949]
  VLA-Pro is a plug-and-play framework designed to enhance cross-task generalization.<n>It stores task-relevant procedural memories at training time and transferring these memories during inference.<n>Experiments on RoboTwin, RLBench, and real-world manipulation tasks show that VLA-Pro consistently improves cross-task generalization.
  arXiv  Detail & Related papers  (2026-05-28T08:14:08Z)
• HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policy [61.668591984635846]
  HAMLET is a framework to adapt Vision-Language-Action models to attend to the historical context during action prediction.<n>We show that HAMLET successfully transforms a state-of-the-art VLA into a history-aware policy.<n>On top of GR00T N1.5, HAMLET achieves an average success rate of 76.4% on history-dependent real-world tasks.
  arXiv  Detail & Related papers  (2025-10-01T09:15:52Z)
• FindingDory: A Benchmark to Evaluate Memory in Embodied Agents [49.18498389833308]
  We introduce a new benchmark for long-range embodied tasks in the Habitat simulator.<n>This benchmark evaluates memory-based capabilities across 60 tasks requiring sustained engagement and contextual awareness.
  arXiv  Detail & Related papers  (2025-06-18T17:06:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.