VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

• URL: http://arxiv.org/abs/2606.21386v1
• Date: Fri, 19 Jun 2026 12:51:21 GMT
• Title: VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models
• Abstract summary: Vision-language--action models (VLAs) achieve state-of-the-art performance on many robotic tasks, yet they can still behave unpredictably in out-of-distribution scenarios.<n>We propose VLA-FAIL, a framework that combines two novel failure detectors with minimal overhead, without requiring failure data.
• Score: 19.137291311347788
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language-action models (VLAs) achieve state-of-the-art performance on many robotic manipulation tasks, yet they can still behave unpredictably in out-of-distribution scenarios. Runtime failure detection is therefore essential for the safe real-world deployment of VLAs. However, existing task failure detectors require computationally expensive action sampling, are based on architectural assumptions that limit their applicability to VLAs, or need access to failure rollouts. We propose VLA-FAIL, a lightweight and broadly applicable failure detection framework for VLAs that combines two novel failure detectors with minimal overhead, without requiring failure data. The first, last-layer Mahalanobis distance (LLMD), detects out-of-distribution states by measuring token-wise deviations in last-layer features relative to the training data. The second, action chunk consistency (ACC), exploits the temporal overlap induced by receding-horizon control and detects failures when consecutive action chunks become inconsistent. To capture the trade-off between detection accuracy and detection latency, we introduce AUCPDT, a threshold-independent metric that jointly evaluates precision, recall, and detection time. Through extensive real-world and simulation experiments, we demonstrate that LLMD and ACC capture complementary failure modes whose combination enables reliable and early failure detection across diverse tasks, frequently outperforming significantly more expensive baseline methods.

Related papers

• FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Models [12.030216781017558]
  Vision-language--action (VLA) policies can fail unpredictably during long-horizon execution.<n>We propose a data-efficient framework that first leverages unlabeled VLA action chunks to construct weak supervision signals.<n>We then use active learning to select only the most uncertain trajectories for timestamp-level annotation.
  arXiv  Detail & Related papers  (2026-09-03T00:04:06Z)
• TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes [65.10128145059127]
  This work introduces Configured Failure Trapping, a novel backdoor attack task against Vision-Language-Action (VLA) models.<n>We propose an effective data engine for synthesizing high-quality target trajectories and an automated suite for measuring configured-failure fidelity.<n>Experiments show that TrapVLA effectively injects configured failure modes into VLA models while largely preserving performance on clean data.
  arXiv  Detail & Related papers  (2026-08-27T03:44:49Z)
• Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents [58.49879338545782]
  Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored.<n>We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model.<n>Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-06-22T09:32:28Z)
• Uncertainty Quantification for Flow-Based Vision-Language-Action Models [33.28454469934064]
  Vision-language-action models (VLAs) combine vision-language backbones with expressive generative action heads trained via flow matching on large-scale robotic datasets.<n>Despite their strong empirical performance in robotic manipulation, VLAs lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable.<n>We propose SAVE, a framework for uncertainty-guided active fine-tuning that reduces the number of costly expert demonstrations required to adapt VLAs to new tasks.
  arXiv  Detail & Related papers  (2026-06-16T15:19:09Z)
• Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring [35.34120853605602]
  textbf-and-Seek is a framework that formulates VLA failure detection as a coarsely supervised learning problem.<n>We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies.
  arXiv  Detail & Related papers  (2026-05-29T04:40:12Z)
• Failure Identification in Imitation Learning Via Statistical and Semantic Filtering [2.2515832864485024]
  Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments.<n>We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module.<n>We also introduce BotFails, a multimodal dataset of real-world tasks for failure detection in robotics.
  arXiv  Detail & Related papers  (2026-04-15T12:27:32Z)
• From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models [42.553888395227766]
  We propose a lightweight, training-free framework for vision-language-action (VLA) models.<n>VLA-SCT operates as a self-correcting control loop, combining data-driven action refinement with conditional logic for termination.<n>Our method achieves consistent improvements across all datasets in the LIBERO benchmark.
  arXiv  Detail & Related papers  (2026-02-02T08:44:40Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• Contamination Detection for VLMs using Multi-Modal Semantic Perturbation [73.76465227729818]
  Open-source Vision-Language Models (VLMs) have achieved state-of-the-art performance on benchmark tasks.<n>Pretraining corpora raise a critical concern for both practitioners and users: inflated performance due to test-set leakage.<n>We show that existing detection approaches either fail outright or exhibit inconsistent behavior.<n>We propose a novel simple yet effective detection method based on multi-modal semantic perturbation.
  arXiv  Detail & Related papers  (2025-11-05T18:59:52Z)
• SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model [52.47816604709358]
  Video anomaly detection (VAD) aims to identify unexpected events in videos and has wide applications in safety-critical domains.<n> vision-language models (VLMs) have demonstrated strong multimodal reasoning capabilities, offering new opportunities for anomaly detection.<n>We propose SlowFastVAD, a hybrid framework that integrates a fast anomaly detector with a slow anomaly detector.
  arXiv  Detail & Related papers  (2025-04-14T15:30:03Z)
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection [56.66677293607114]
  We propose Code-as-Monitor (CaM) for both open-set reactive and proactive failure detection.<n>To enhance the accuracy and efficiency of monitoring, we introduce constraint elements that abstract constraint-related entities.<n>Experiments show that CaM achieves a 28.7% higher success rate and reduces execution time by 31.8% under severe disturbances.
  arXiv  Detail & Related papers  (2024-12-05T18:58:27Z)
• Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress [31.952925824381325]
  We propose a runtime monitoring framework that splits the detection of failures into two complementary categories. We use Vision Language Models (VLMs) to detect when the policy confidently and consistently takes actions that do not solve the task. By unifying temporal consistency detection and VLM runtime monitoring, Sentinel detects 18% more failures than using either of the two detectors alone.
  arXiv  Detail & Related papers  (2024-10-06T22:13:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.