Resolving State Ambiguity in Robot Manipulation via Adaptive Working Memory Recoding

• URL: http://arxiv.org/abs/2512.24638v1
• Date: Wed, 31 Dec 2025 05:20:13 GMT
• Title: Resolving State Ambiguity in Robot Manipulation via Adaptive Working Memory Recoding
• Abstract summary: PAM is a novel visuomotor policy equipped with Adaptive working Memory.<n>It supports a 300-frame history window while maintaining high inference speed.<n>We meticulously design 7 tasks and verify that PAM can handle multiple scenarios of state ambiguity simultaneously.
• Score: 12.400431805327093
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State ambiguity is common in robotic manipulation. Identical observations may correspond to multiple valid behavior trajectories. The visuomotor policy must correctly extract the appropriate types and levels of information from the history to identify the current task phase. However, naively extending the history window is computationally expensive and may cause severe overfitting. Inspired by the continuous nature of human reasoning and the recoding of working memory, we introduce PAM, a novel visuomotor Policy equipped with Adaptive working Memory. With minimal additional training cost in a two-stage manner, PAM supports a 300-frame history window while maintaining high inference speed. Specifically, a hierarchical frame feature extractor yields two distinct representations for motion primitives and temporal disambiguation. For compact representation, a context router with range-specific queries is employed to produce compact context features across multiple history lengths. And an auxiliary objective of reconstructing historical information is introduced to ensure that the context router acts as an effective bottleneck. We meticulously design 7 tasks and verify that PAM can handle multiple scenarios of state ambiguity simultaneously. With a history window of approximately 10 seconds, PAM still supports stable training and maintains inference speeds above 20Hz. Project website: https://tinda24.github.io/pam/

Related papers

• TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulation [14.84720789511593]
  We present TemporalFlow-VLA, which learns compact execution history through physically grounded temporal supervision.<n>We show that TemporalFlow-VLA achieves 97.63 +/- 0.26% average success on LIBERO, including 96.60 +/- 0.87% on LIBERO Long, and 85.5%/84.2% Clean/Randomized success across 12 RoboTwin tasks.
  arXiv  Detail & Related papers  (2026-08-27T09:00:56Z)
• Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering [93.80043825360514]
  Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes.<n>We investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries.<n>We find that simply preserving existing memory is often insufficient.<n>We highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation.
  arXiv  Detail & Related papers  (2026-07-23T17:50:45Z)
• WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory [50.19449886428936]
  We introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory.<n>The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning.<n>Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following, and in-context adaptation.
  arXiv  Detail & Related papers  (2026-07-21T08:25:37Z)
• Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control [33.5619212312672]
  General-purpose robots operating in partially observable environments, such as homes, require memory to support autonomy.<n>We introduce HALO, a visuomotor policy with an attention-based memory retrieval mechanism for long-horizon control.
  arXiv  Detail & Related papers  (2026-06-23T20:07:23Z)
• Scaling Short-Term Memory of Visuomotor Policies for Long-Horizon Tasks [69.19366746169906]
  We present PRISM, a transformer-based architecture for visuomotor policies to use short-term memory.<n>Piracy filters retrieved information to suppress irrelevant details, improving performance.<n> hierarchical architecture compresses local information into compact tokens, improving its compute and memory footprint.
  arXiv  Detail & Related papers  (2026-06-15T03:49:23Z)
• MemoAct: Atkinson-Shiffrin-Inspired Memory-Augmented Visuomotor Policy for Robotic Manipulation [6.490934654648497]
  MemoAct is a hierarchical memory-based policy that leverages distinct memory tiers to tackle specific bottlenecks.<n>MemoAct achieves superior performance compared to both existing Markovian baselines and history-aware policies.
  arXiv  Detail & Related papers  (2026-03-19T05:02:43Z)
• VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory [31.464584758455356]
  VPWEM is a non-Markovian visuomotor policy equipped with working and episodic memories.<n>It exploits both short-term and episode-wide information for action generation with nearly constant memory and computation per step.
  arXiv  Detail & Related papers  (2026-03-05T07:52:50Z)
• Non-Markovian Long-Horizon Robot Manipulation via Keyframe Chaining [56.62125584296097]
  Keyframe-Chaining VLA is a framework that extracts and links key historical frames to model long-horizon dependencies.<n>We design a progress-aware mechanism that dynamically retrieves historical frames based on their temporal relevance to the current execution phase.<n>We introduce a suite of four Non-Markovian manipulation tasks built upon the ManiSkill simulator to measure task success rates.
  arXiv  Detail & Related papers  (2026-03-02T05:26:29Z)
• STaR: Scalable Task-Conditioned Retrieval for Long-Horizon Multimodal Robot Memory [16.76678779219906]
  Mobile robots are often deployed over long durations in diverse open, dynamic scenes.<n>A core challenge is to build a scalable long-horizon memory that supports an agentic workflow for planning, retrieval, and reasoning.<n>We present STaR, a task-agnostic, multimodal long-term memory that generalizes to unseen queries.
  arXiv  Detail & Related papers  (2026-02-09T22:38:53Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Quantifying Memory Use in Reinforcement Learning with Temporal Range [51.98491034847041]
  Temporal Range is a model-agnostic metric that treats first-order sensitivities of multiple vector outputs across a temporal window to the input sequence as a temporal influence profile.<n>We also report Temporal Range for a compact Long Expressive Memory (LEM) policy trained on the task, using it as a proxy readout of task-level memory.
  arXiv  Detail & Related papers  (2025-12-05T22:58:09Z)
• MemER: Scaling Up Memory for Robot Control via Experience Retrieval [46.5398413633767]
  Humans routinely rely on memory to tasks, yet most robot policies lack this capability.<n>We propose a hierarchical policy framework, where the high-level policy is trained to select and track previous relevants from its experience.<n>Our approach, MemER, outperforms prior methods on three real-world long-horizon robotic manipulation tasks that require minutes of memory.
  arXiv  Detail & Related papers  (2025-10-23T08:26:17Z)
• Memo: Training Memory-Efficient Embodied Agents with Reinforcement Learning [53.72709564555407]
  Memo is a transformer-based architecture and training recipe for reinforcement learning.<n>It incorporates the creation and retrieval of memory by interleaving periodic summarization tokens with the inputs of a model during training.<n>We demonstrate Memo's effectiveness on a gridworld meta-RL benchmark and a multi-object navigation task in photo-realistic indoor settings.
  arXiv  Detail & Related papers  (2025-10-22T16:24:47Z)
• Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs.<n>Specifically, we introduce a Ghost Spatial Masking (GSM) module, embedded within a Transformer encoder, for spatial feature extraction.<n>We benchmark three practical sports datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.