TFP: Temporally Conditioned Memory-Fusion Policies for Visuomotor Learning

• URL: http://arxiv.org/abs/2607.08283v2
• Date: Wed, 15 Jul 2026 15:46:02 GMT
• Title: TFP: Temporally Conditioned Memory-Fusion Policies for Visuomotor Learning
• Abstract summary: Vision--Language--Action (VLA) policies such as $_0.5$ and OpenVLA perform well on many tasks, but they are often reactive.<n>We argue that such tasks require not only memory, but dynamics-aware belief updates.
• Score: 11.149498165285491
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision--Language--Action (VLA) policies such as $π_{0.5}$ and OpenVLA perform well on many manipulation tasks, but they are often reactive: the next action is predicted from the current observation, instruction, and proprioceptive state. This assumption breaks down in stage-dependent manipulation, where visually similar states may require different actions depending on latent task progress and previous interaction outcomes. We argue that such tasks require not only memory, but dynamics-aware belief updates: the policy should preserve task progress during stable or occluded phases and revise its belief near contact, release, or subgoal transitions. We introduce Temporally Conditioned Memory-Fusion Policies (TFP), a lightweight memory-action framework for VLA backbones. TFP maintains an episode-local task-progress belief with Liquid Time-Constant dynamics and injects the updated belief directly into the flow-matching action decoder through adaptive modulation. This lets temporally accumulated context shape the generated action chunk, rather than serving only as passive history context. With a 3.3B-parameter model, TFP improves the average success rate from \(96.9\%\) to \(98.75\%\) on LIBERO and from \(91.4\%\) to \(93.77\%\) on LIBERO-plus. On the memory-focused MIKASA ShellGameTouch diagnostic, TFP achieves success up to \(75.0\%\). Mechanistic analyses show that write-gain changes near manipulation events are about \(6\times\) larger than far non-event phases, and hidden-state interventions show that the belief causally modulates generated action chunks. These results suggest that compact, event-sensitive memory dynamics can improve VLA policies under occlusion, visual perturbation, and stage-dependent task structure.

Related papers

• AGM: Achievement-Grounded Memory for Closed-Loop Agents with Frozen VLA Policies [17.09836507714724]
  We propose Achievement-Grounded Memory (AGM) for frozen vision-language-action (VLA) policies.<n>AGM represents a task as a subgoal sequence with a progress pointer and advances this memory only after the current subgoal is verified by physical evidence.<n>On the RoboMME Counting benchmark, AGM reaches on PickXTimes and on BinFill, surpassing the strongest memory-augmented baseline by points on average.
  arXiv  Detail & Related papers  (2026-08-30T03:50:49Z)
• Retrieve in Time, Correct in Frequency [14.108343620177314]
  We introduce Retrieve in Time, Correct in Frequency (RTCF), a training-free test-time correction framework.<n>RTCF separates which experience to retrieve from which part of its action to transfer.<n>It raises aggregate success from 86.4% to 88.4% and improves LIBERO-Long from 61.6% to 68.6%.
  arXiv  Detail & Related papers  (2026-08-05T06:58:00Z)
• FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking [51.97839311685636]
  FutureRTC is a plug-and-play adaptation framework that predicts execution-time observations and states for asynchronous VLA control without modifying the underlying policy.<n>We show that FutureRTC achieves robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.
  arXiv  Detail & Related papers  (2026-07-27T05:10:16Z)
• KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policies [14.253600207124194]
  Memory can address this ambiguity by enabling policies to infer task progress from execution history.<n>We propose KEMO, a lightweight plug-in memory framework that automatically preserves kinematics associated with task-relevant state changes for VLA policies.<n>We evaluate KEMO on various real-world dualarm manipulation tasks spanning 2 to 6 scored subtasks, and trajectory length ranging from 830 steps to 2846 execution steps (durations from 28 to 95 seconds)
  arXiv  Detail & Related papers  (2026-06-22T16:57:43Z)
• EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies [68.812675280427]
  EventVLA is an end-to-end framework founded on the concept of sparse visual evidence memory.<n>KEM directly predicts future probabilities from the VLA's latent embeddings to autonomously capture and store sparse, task-critical visual events.<n>We propose RoboTwin-MeM, a diagnostic benchmark designed to evaluate non-Markovian manipulation tasks with interactive visual evidence.
  arXiv  Detail & Related papers  (2026-06-18T11:11:37Z)
• Planning-aligned Token Compression for Long-Context Autonomous Driving [95.59023657139208]
  We propose a planning-aligned working memory framework built on conditional VQ-VA.<n>Compression is conditioned on both historical trajectory and a learned planning intent.<n>We evaluate on high-signal dynamic scenarios where historical context is most critical for behavior correctness.
  arXiv  Detail & Related papers  (2026-06-05T17:16:21Z)
• Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation [5.524438437366922]
  We present a phase-conditioned, force-aware framework for deformable object manipulation.<n>A FiLM-conditioned ACT encoder modulates feature extraction based on the current task phase.<n>A multi-modal phase predictor fusing visual, force, and pose feedback estimates the phase in real time, detecting contact failures that are invisible to vision alone.
  arXiv  Detail & Related papers  (2026-05-28T05:59:53Z)
• From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model [76.88595728131288]
  Vision-Language-Action (VLA) models often suffer from performance degradation under distribution shifts.<n>We propose textbfBehaviorVLA, a framework that facilitates robust manipulation through the learning of a temporally coherent behavioral representations.<n> Experiments on RoboTwin 2.0, LIBERO, and CALVIN demonstrate state-of-the-art success rates of 58%, 98%, and 4.36 (Avg.Len)
  arXiv  Detail & Related papers  (2026-05-21T16:14:19Z)
• OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulation [55.56490813640669]
  VLA models typically act only on the current frame, while future prediction and object-aware reasoning are often learned in separate latent spaces.<n>We propose OFlow, a framework that addresses both limitations by unifying temporal foresight and object-aware reasoning in a shared semantic latent space.<n>Our method forecasts future latents with temporal flow matching, factorizes them into object-aware representations that emphasize physically relevant cues.
  arXiv  Detail & Related papers  (2026-04-20T06:38:01Z)
• VLA-IAP: Training-Free Visual Token Pruning via Interaction Alignment for Vision-Language-Action Models [27.12266806191131]
  Vision-Language-Action (VLA) models have rapidly advanced embodied intelligence, enabling robots to execute complex, instruction-driven tasks.<n>Current approaches often prune visually sparse yet structurally critical regions that support manipulation, leading to unstable behavior during early task phases.<n>Our proposed textbftraining-free method, VLA-IAP (Interaction-Aligned Pruning), introduces a geometric prior mechanism to preserve structural anchors and a dynamic scheduling strategy.
  arXiv  Detail & Related papers  (2026-03-24T09:33:05Z)
• VPWEM: Non-Markovian Visuomotor Policy with Working and Episodic Memory [31.464584758455356]
  VPWEM is a non-Markovian visuomotor policy equipped with working and episodic memories.<n>It exploits both short-term and episode-wide information for action generation with nearly constant memory and computation per step.
  arXiv  Detail & Related papers  (2026-03-05T07:52:50Z)
• Gated Differentiable Working Memory for Long-Context Language Modeling [80.27483324685434]
  We propose Gdwm (Gated Differentiable Working Memory), a framework that introduces a write controller to gate the consolidation process.<n>Experiments on ZeroSCROLLS and LongBench v2 demonstrate that Gdwm achieves comparable or superior performance with 4$times$ fewer gradient steps than uniform baselines.
  arXiv  Detail & Related papers  (2026-01-19T10:00:33Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling [84.51372201195132]
  CronusVLA is a unified framework that extends single-frame VLA models to the multi-frame paradigm.<n>CronusVLA achieves leading performance and superior robustness, with a 70.9% success rate.<n>These results highlight the potential of efficient multi-frame adaptation in VLA models for more powerful and robust real-world deployment.
  arXiv  Detail & Related papers  (2025-06-24T17:30:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.