Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2608.14822v1
• Date: Fri, 14 Aug 2026 18:49:46 GMT
• Title: Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models
• Abstract summary: Vision-language-action (VLA) models are fragile to online disruptions.<n>We propose Counterfactual Realignment (CoRe), a training-free framework that recovers a frozen VLA at inference time without failure data.
• Score: 10.832654624297135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models have improved the flexibility and generality of robotic manipulation, yet they remain fragile to online disruptions, such as changes in task goal, scene configuration, or robot state. Existing recovery methods often require failure data, policy retraining, or external corrective agents, introducing additional data requirements and execution risks. We propose Counterfactual Realignment (CoRe), a training-free framework that recovers a frozen VLA at inference time without failure data. Upon detecting a deviation, CoRe imagines how the policy would continue toward the current goal from a recent viable state, using synthesized observations in place of physical execution, and then minimally realigns the robot and scene to rejoin this imagined continuation before returning control to the policy. Recovery is therefore planned without physical trial-and-error, preserves completed task progress, and handles both mid-episode instruction changes and physical perturbations in a unified manner. Extensive experiments across multiple simulators, VLA backbones, and real-world settings show that CoRe improves success rates by up to 85.0 percentage points to near-nominal levels while reducing physical restorations by 42.2%, without policy fine-tuning or failure-specific recovery training.

Related papers

• REVOLVE: An Automated Closed-Loop Framework for Evolving Robot Manipulation with Minimal Human Intervention [17.399763167916195]
  We present REVOLVE, an automated closed-loop framework for evolving robot manipulation with minimal human intervention.<n>Built on a unified software platform, REVOLVE integrates data collection, policy training and deployment, failure recovery, and continual learning into a single closed-loop workflow.
  arXiv  Detail & Related papers  (2026-09-13T16:08:03Z)
• FLARE: A Failure-Aware Framework for Autonomous Correction and Recovery in Visual-Language Robotic Manipulation [43.89036551517797]
  Vision-Language-Action Models (VLAs) have demonstrated significant promise in generalizing to complex, long-horizon robotic manipulation tasks.<n>Their performance remains brittle, as they are typically trained on trajectory-monotonic, failure-free demonstrations.<n>We propose FLARE, a novel framework that endows VLAs with robust error recovery capabilities through a Retry" and Reset" paradigm.
  arXiv  Detail & Related papers  (2026-08-27T05:58:06Z)
• EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration [75.13534797264485]
  We show that egocentric human data capturing failure recovery processes provides a scalable alternative to robot teleoperation.<n>By efficiently arranging task-level failure configurations and recording short recovery segments, human operators can generate more than 10x as much valid recovery data per hour.
  arXiv  Detail & Related papers  (2026-07-22T04:44:26Z)
• FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement [10.94594187411586]
  Failure-Aware Retry (FAR) is a framework that enables robots to learn from previous failures at test time, adapt their behavior accordingly, and eventually complete the task autonomously.<n>FAR substantially improves success rates and robustness, with average gains of 17.6% over the standard diffusion policy in simulation and 11.7% in the real world.
  arXiv  Detail & Related papers  (2026-07-01T16:01:54Z)
• Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement [52.93267443851685]
  Vision-Language-Action (VLA) models can generalize across diverse manipulation tasks.<n>Can a reinforcement learning policy trained purely in simulation improve the robustness of real-world VLAs zero-shot?<n>We propose an object-centric residual RL framework that refines VLA actions using object poses.
  arXiv  Detail & Related papers  (2026-06-17T11:36:54Z)
• Back to the Familiar Future: Failure Recovery for VLA Policies via Pre-Imagined Milestone Selection [19.954434707400704]
  We propose Back to the Familiar Future (B2FF), a recovery framework for foresight-driven vision-language-action (VLA) policies.<n>Before execution, the VLA generates a milestone bank of familiar future states conditioned on the clean initial observation.<n>At recovery time, a recoverability-aware selector selects a recovery milestone from this bank and enforces it as a fixed visual goal.
  arXiv  Detail & Related papers  (2026-06-08T09:30:38Z)
• RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models [90.39703013636868]
  RePO-VLA is a recovery-driven policy optimization framework.<n>It assigns distinct roles to success, recovery, and failure trajectories.<n>It raises adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials.
  arXiv  Detail & Related papers  (2026-05-10T08:24:05Z)
• ARES: Scalable and Practical Gradient Inversion Attack in Federated Learning through Activation Recovery [73.8181449261685]
  Federated Learning (FL) enables collaborative model training by sharing model updates instead of raw data, aiming to protect user privacy.<n>Recent studies reveal that these shared updates can inadvertently leak sensitive training data through gradient inversion attacks (GIAs)
  arXiv  Detail & Related papers  (2026-03-18T11:40:44Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• RecoveryChaining: Learning Local Recovery Policies for Robust Manipulation [41.38308130776887]
  We propose to use hierarchical reinforcement learning to learn a recovery policy.<n>The recovery policy is triggered when a failure is detected based on sensory observations.<n>We successfully transfer recovery policies learned in simulation to a physical robot to demonstrate the feasibility of sim-to-real transfer.
  arXiv  Detail & Related papers  (2024-10-17T19:14:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.