Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection

• URL: http://arxiv.org/abs/2606.09258v1
• Date: Mon, 08 Jun 2026 09:30:38 GMT
• Title: Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection
• Abstract summary: We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven vision-language-action (VLA) policies.<n>Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation.<n>At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal.
• Score: 19.954434707400704
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) policies can deviate from nominal trajectories during manipulation, even when tasks remain physically feasible. Recovering from these deviations is challenging, as they push the policy into unfamiliar state spaces where direct re-planning frequently destabilizes action sequences. We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven VLAs that leverages future visual conditioning as a recovery interface. Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation. At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal. This enables the VLA to robustly map off-trajectory observations back to a familiar future. On failure-injected LIBERO, under controlled recovery timing aligned with the injected failure, B2FF increases the average success rate of a baseline VLA from 56.3% to 74.0%, demonstrating that pre-imagined milestones can guide recovery without fine-tuning the low-level action generator.

Related papers

• Reuse Before You Retrieve: Diagnosing Headroom and Complementarity for Test-Time Augmentation of Embodied Multimodal Policies [56.86456077356948]
  Frozen-language-action (VLA) policies are increasingly improved at test time.<n>Additional sampling is useful only when better behavior already exists within the policy's rollouts.<n>Retrieval is most useful when the relevant action prior is not reliably represented by the policy.
  arXiv  Detail & Related papers  (2026-08-18T08:11:31Z)
• Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models [10.832654624297135]
  Vision-language-action (VLA) models are fragile to online disruptions.<n>We propose Counterfactual Realignment (CoRe), a training-free framework that recovers a frozen VLA at inference time without failure data.
  arXiv  Detail & Related papers  (2026-08-14T18:49:46Z)
• ActFovea: Runtime Safeguarding for VLA Policies via Spatiotemporal Visual-Action Consistency [23.6841289206823]
  Vision-language-action (VLA) policies achieve strong performance in robotic manipulation.<n>They are vulnerable to runtime disturbances that break the temporal alignment among visual observations, robot states, and executed actions.<n>We introduce ActFovea, a plug-and-play safeguarding framework that detects and mitigates such failures without retraining or modifying the underlying VLA policy.
  arXiv  Detail & Related papers  (2026-07-31T08:47:57Z)
• ReCoVLA: VLM-Guided Reward Compilation for Failure Recovery in Vision-Language-Action Policies [16.305613129361593]
  Vision-language-action (VLA) policies provide strong priors for language-conditioned manipulation, but remain brittle in off-nominal states requiring targeted recovery.<n>We propose ReCoVLA -- a failure-conditioned residual recovery framework that keeps a pretrained VLA policy frozen.
  arXiv  Detail & Related papers  (2026-06-08T15:29:09Z)
• RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models [90.39703013636868]
  RePO-VLA is a recovery-driven policy optimization framework.<n>It assigns distinct roles to success, recovery, and failure trajectories.<n>It raises adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials.
  arXiv  Detail & Related papers  (2026-05-10T08:24:05Z)
• Learning from Mistakes: Post-Training for Driving VLA with Takeover Data [22.44026866655379]
  TakeVLA is a novel VLA post-training framework that overcomes shortcomings through two complementary innovations.<n>First, we introduce pre-takeover language supervision, which allows the VLA to learn from mistakes proactively.<n>Second, we propose Scenario Dreaming, a reinforcement fine-tuning paradigm that operates in reconstruceted takeover scenarios.
  arXiv  Detail & Related papers  (2026-03-16T08:33:48Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• Recursive Belief Vision Language Action Models [0.0]
  Long-horizon manipulation requires persistent, action-conditioned state representations.<n>Current vision-language models exhibit limited temporal and physical reasoning.<n>This paper introduces RB-VLA, a belief-centric architecture trained with self-supervised world-model objectives.
  arXiv  Detail & Related papers  (2026-02-24T08:02:16Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding [60.06899554269808]
  CycleVLA is a system that equips Vision-Language-Action models (VLAs) with proactive self-correction.<n>CycleVLA achieves this by integrating a progress-aware VLA that flags critical subtask transition points.<n>Extensive experiments show that CycleVLA improves performance for both well-trained and under-trained VLAs.
  arXiv  Detail & Related papers  (2026-01-05T17:31:01Z)
• Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning [71.19675094463834]
  This work introduces Counterfactual VLA, a self-reflective VLA framework that enables the model to reason about and revise its planned actions before execution.<n>CF-VLA first generates time-segmented meta-actions that summarize driving intent, and then performs counterfactual reasoning conditioned on both the meta-actions and the visual context.<n> Experiments on large-scale driving datasets show that CF-VLA improves trajectory accuracy by up to 17.6%, enhances safety metrics by 20.5%, and exhibits adaptive thinking.
  arXiv  Detail & Related papers  (2025-12-30T19:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.