FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2609.04277v1
• Date: Thu, 03 Sep 2026 00:04:06 GMT
• Title: FailureSpot: Label-Efficient Timestamp-Level Failure Detection for Vision-Language-Action Models
• Abstract summary: Vision-language--action (VLA) policies can fail unpredictably during long-horizon execution.<n>We propose a data-efficient framework that first leverages unlabeled VLA action chunks to construct weak supervision signals.<n>We then use active learning to select only the most uncertain trajectories for timestamp-level annotation.
• Score: 12.030216781017558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) policies have shown strong potential for general-purpose robotic manipulation, but they can still fail unpredictably during long-horizon execution, making reliable failure detection essential for safe deployment. Existing methods either rely on visual models that typically detect failures only after erroneous actions have occurred, or use lightweight proactive detectors trained on VLA internal representations. However, these proactive methods are often supervised with trajectory-level labels, causing normal pre-failure behavior in unsuccessful trajectories to be incorrectly labeled as failure. This supervision mismatch introduces label noise and limits both trajectory-level detection accuracy and precise timestamp-level failure localization. In this work, we study fine-grained timestamp-level VLA failure detection while addressing the cost of dense annotation. We propose a data-efficient framework that first leverages unlabeled VLA action chunks to construct action-derived weak supervision signals, capturing abnormal patterns such as inconsistent consecutive chunks, frozen or idle actions, and aggressive random motions. We then use active learning to select only the most uncertain trajectories for timestamp-level annotation and fine-tune the detector with these informative labels. Experiments across multiple VLA policies show that our method improves both timestamp-level and trajectory-level failure detection performance.

Related papers

• TrapVLA: Trapping Vision-Language-Action Models in Configured Failure Modes [65.10128145059127]
  This work introduces Configured Failure Trapping, a novel backdoor attack task against Vision-Language-Action (VLA) models.<n>We propose an effective data engine for synthesizing high-quality target trajectories and an automated suite for measuring configured-failure fidelity.<n>Experiments show that TrapVLA effectively injects configured failure modes into VLA models while largely preserving performance on clean data.
  arXiv  Detail & Related papers  (2026-08-27T03:44:49Z)
• Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents [58.49879338545782]
  Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored.<n>We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model.<n>Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-06-22T09:32:28Z)
• VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models [19.137291311347788]
  Vision-language--action models (VLAs) achieve state-of-the-art performance on many robotic tasks, yet they can still behave unpredictably in out-of-distribution scenarios.<n>We propose VLA-FAIL, a framework that combines two novel failure detectors with minimal overhead, without requiring failure data.
  arXiv  Detail & Related papers  (2026-06-19T12:51:21Z)
• Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring [35.34120853605602]
  textbf-and-Seek is a framework that formulates VLA failure detection as a coarsely supervised learning problem.<n>We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies.
  arXiv  Detail & Related papers  (2026-05-29T04:40:12Z)
• Failure Identification in Imitation Learning Via Statistical and Semantic Filtering [2.2515832864485024]
  Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments.<n>We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module.<n>We also introduce BotFails, a multimodal dataset of real-world tasks for failure detection in robotics.
  arXiv  Detail & Related papers  (2026-04-15T12:27:32Z)
• TAG: Target-Agnostic Guidance for Stable Object-Centric Inference in Vision-Language-Action Models [59.13964209628383]
  Vision-Language--Action (VLA) policies have shown strong progress in mapping language instructions and visual observations to robotic actions, yet their reliability degrades in cluttered scenes with distractors.<n>We propose TAG (Target-Agnostic Guidance), a simple inference-time guidance mechanism that explicitly reduces distractor- and appearance-induced bias in VLA policies.<n>We evaluate TAG on standard manipulation benchmarks, including LIBERO, LIBERO-Plus, and VLABench, where it consistently improves robustness under clutter and reduces near-miss and wrong-object executions.
  arXiv  Detail & Related papers  (2026-03-25T17:56:32Z)
• CycleVLA: Proactive Self-Correcting Vision-Language-Action Models via Subtask Backtracking and Minimum Bayes Risk Decoding [60.06899554269808]
  CycleVLA is a system that equips Vision-Language-Action models (VLAs) with proactive self-correction.<n>CycleVLA achieves this by integrating a progress-aware VLA that flags critical subtask transition points.<n>Extensive experiments show that CycleVLA improves performance for both well-trained and under-trained VLAs.
  arXiv  Detail & Related papers  (2026-01-05T17:31:01Z)
• Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress [31.952925824381325]
  We propose a runtime monitoring framework that splits the detection of failures into two complementary categories. We use Vision Language Models (VLMs) to detect when the policy confidently and consistently takes actions that do not solve the task. By unifying temporal consistency detection and VLM runtime monitoring, Sentinel detects 18% more failures than using either of the two detectors alone.
  arXiv  Detail & Related papers  (2024-10-06T22:13:30Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.