Failure Identification in Imitation Learning Via Statistical and Semantic Filtering

• URL: http://arxiv.org/abs/2604.13788v1
• Date: Wed, 15 Apr 2026 12:27:32 GMT
• Title: Failure Identification in Imitation Learning Via Statistical and Semantic Filtering
• Abstract summary: Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments.<n>We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module.<n>We also introduce BotFails, a multimodal dataset of real-world tasks for failure detection in robotics.
• Score: 2.2515832864485024
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments: rare events such as hardware faults, defective parts, unexpected human actions, or any state that lies outside the training distribution can lead to failed executions. Vision-based Anomaly Detection (AD) methods emerged as an appropriate solution to detect these anomalous failure states but do not distinguish failures from benign deviations. We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module. Leveraging recent AD methods, FIDeL builds a compact representation of nominal demonstrations and aligns incoming observations via optimal transport matching to produce anomaly scores and heatmaps. Spatio-temporal thresholds are derived with an extension of conformal prediction, and a Vision-Language Model (VLM) performs semantic filtering to discriminate benign anomalies from genuine failures. We also introduce BotFails, a multimodal dataset of real-world tasks for failure detection in robotics. FIDeL consistently outperforms state-of-the-art baselines, yielding +5.30% percent AUROC in anomaly detection and +17.38% percent failure-detection accuracy on BotFails compared to existing methods.

Related papers

• Coverage Aware Active Evaluation for Failure Discovery with Paired Systems [32.870747441157654]
  We propose an adaptive failure discovery method that combines proxy evaluations with limited target system results.<n>Our method discovers up to 2$times$ as many failures as random sampling and active-learning baselines.
  arXiv  Detail & Related papers  (2026-08-13T19:26:25Z)
• Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents [58.49879338545782]
  Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored.<n>We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model.<n>Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-06-22T09:32:28Z)
• VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models [19.137291311347788]
  Vision-language--action models (VLAs) achieve state-of-the-art performance on many robotic tasks, yet they can still behave unpredictably in out-of-distribution scenarios.<n>We propose VLA-FAIL, a framework that combines two novel failure detectors with minimal overhead, without requiring failure data.
  arXiv  Detail & Related papers  (2026-06-19T12:51:21Z)
• Hide-and-Seek in Trajectories: Discovering Failure Signals for VLA Runtime Monitoring [35.34120853605602]
  textbf-and-Seek is a framework that formulates VLA failure detection as a coarsely supervised learning problem.<n>We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies.
  arXiv  Detail & Related papers  (2026-05-29T04:40:12Z)
• Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection [52.5174167737992]
  Video anomaly detection (VAD) aims to identify abnormal events in videos.<n>We propose SteerVAD, which advances MLLM-based VAD by shifting from passively reading to actively steering and rectifying internal representations.<n>Our method achieves state-of-the-art performance among tuning-free approaches requiring only 1% of training data.
  arXiv  Detail & Related papers  (2026-02-27T13:48:50Z)
• ProbeLLM: Automating Principled Diagnosis of LLM Failures [89.44131968886184]
  We propose ProbeLLM, a benchmark-agnostic automated probing framework that elevates weakness discovery from individual failures to structured failure modes.<n>By restricting probing to verifiable test cases and leveraging tool-augmented generation and verification, ProbeLLM grounds failure discovery in reliable evidence.
  arXiv  Detail & Related papers  (2026-02-13T14:33:13Z)
• Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models [53.20969621498248]
  We propose an automatic robot failure synthesis approach that procedurally perturbs successful trajectories to generate diverse planning and execution failures.<n>We construct three new failure detection benchmarks: RLBench-Fail, BridgeDataV2-Fail, and UR5-Fail.<n>We then train Guardian, a VLM with multi-view images for detailed failure reasoning and detection.
  arXiv  Detail & Related papers  (2025-12-01T17:57:27Z)
• Contamination Detection for VLMs using Multi-Modal Semantic Perturbation [73.76465227729818]
  Open-source Vision-Language Models (VLMs) have achieved state-of-the-art performance on benchmark tasks.<n>Pretraining corpora raise a critical concern for both practitioners and users: inflated performance due to test-set leakage.<n>We show that existing detection approaches either fail outright or exhibit inconsistent behavior.<n>We propose a novel simple yet effective detection method based on multi-modal semantic perturbation.
  arXiv  Detail & Related papers  (2025-11-05T18:59:52Z)
• Reliably Detecting Model Failures in Deployment Without Labels [14.069153343960734]
  This paper formalizes and addresses the problem of post-deployment deterioration (PDD) monitoring.<n>We propose D3M, a practical and efficient monitoring algorithm based on the disagreement of predictive models.<n> Empirical results on both standard benchmark and a real-world large-scale internal medicine dataset demonstrate the effectiveness of the framework.
  arXiv  Detail & Related papers  (2025-06-05T13:56:18Z)
• Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies [19.27526590452503]
  FAIL-Detect is a two-stage approach for failure detection in imitation learning-based robotic manipulation.<n>We first distill policy inputs and outputs into scalar signals that correlate with policy failures and capture uncertainty.<n>Our experiments show learned signals to be mostly consistently effective, particularly when using our novel flow-based density estimator.
  arXiv  Detail & Related papers  (2025-03-11T15:47:12Z)
• Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [59.41026558455904]
  We focus on multi-modal anomaly detection. Specifically, we investigate early multi-modal approaches that attempted to utilize models pre-trained on large-scale visual datasets. We propose a Local-to-global Self-supervised Feature Adaptation (LSFA) method to finetune the adaptors and learn task-oriented representation toward anomaly detection.
  arXiv  Detail & Related papers  (2024-01-06T07:30:41Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.