From Mystery to Mastery: Failure Diagnosis for Improving Manipulation Policies

• URL: http://arxiv.org/abs/2412.02818v2
• Date: Sat, 08 Feb 2025 21:54:30 GMT
• Title: From Mystery to Mastery: Failure Diagnosis for Improving Manipulation Policies
• Abstract summary: This paper introduces Robot Manipulation Diagnosis (RoboMD), a systematic framework designed to automatically identify critical failure modes (FMs)<n>Considering the vast space of potential FMs in a pre-trained manipulation policy, we leverage deep reinforcement learning (deep RL) to explore and uncover these FMs.<n>We demonstrate RoboMD's effectiveness in diagnosing unknown failures in unstructured environments.
• Score: 35.18739716338974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robot manipulation policies often fail for unknown reasons, posing significant challenges for real-world deployment. Researchers and engineers typically address these failures using heuristic approaches, which are not only labor-intensive and costly but also prone to overlooking critical failure modes (FMs). This paper introduces Robot Manipulation Diagnosis (RoboMD), a systematic framework designed to automatically identify FMs arising from unanticipated changes in the environment. Considering the vast space of potential FMs in a pre-trained manipulation policy, we leverage deep reinforcement learning (deep RL) to explore and uncover these FMs using a specially trained vision-language embedding that encodes a notion of failures. This approach enables users to probabilistically quantify and rank failures in previously unseen environmental conditions. Through extensive experiments across various manipulation tasks and algorithms, we demonstrate RoboMD's effectiveness in diagnosing unknown failures in unstructured environments, providing a systematic pathway to improve the robustness of manipulation policies.

Related papers

• Don't Fool Me Twice: Adapting to Adversity in the Wild with Experience-Driven Reasoning [2.3555777010480337]
  In robotics, dangers and adversity modes are often embodiment-specific and relative to each agent.<n>We propose a continual learning framework for a mobile embodied agent to learn from disturbances and attribute anomalous behaviours to causes.<n>We present four hypotheses and validate them in simulation and on hardware across embodiments and adversity modes.
  arXiv  Detail & Related papers  (2026-05-29T10:30:02Z)
• Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration [53.772014300855375]
  We introduce NIAgent, a multi-agent system for autonomous end-to-end neuroimaging analysis.<n>Unlike conventional flat tool-calling agents, NIAgent adopts a code-centric execution paradigm.<n>We propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection.
  arXiv  Detail & Related papers  (2026-05-10T06:30:19Z)
• MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems [38.44649280816596]
  We propose MAS-FIRE, a systematic framework for fault injection and reliability evaluation of Multi-Agent Systems.<n>We define a taxonomy of 15 fault types covering intra-agent cognitive errors and inter-agent coordination failures.<n>Applying MAS-FIRE to three representative MAS architectures, we uncover a rich set of fault-tolerant behaviors.
  arXiv  Detail & Related papers  (2026-02-23T13:47:43Z)
• SOE: Sample-Efficient Robot Policy Self-Improvement via On-Manifold Exploration [58.05143960563826]
  On-Manifold Exploration (SOE) is a framework that enhances policy exploration and improvement in robotic manipulation.<n>SOE learns a compact latent representation of task-relevant factors and constrains exploration to the manifold of valid actions.<n>It can be seamlessly integrated with arbitrary policy models as a plug-in module, augmenting exploration without degrading the base policy performance.
  arXiv  Detail & Related papers  (2025-09-23T17:54:47Z)
• An Adversarial-Driven Experimental Study on Deep Learning for RF Fingerprinting [6.503988096115075]
  Radio frequency (RF) fingerprinting has emerged as a promising physical-layer device identification mechanism.<n>Deep learning (DL) methods have demonstrated state-of-the-art performance in this domain.<n>We show that training DL models on raw received signals causes the models to entangle RF fingerprints with environmental and signal-pattern features.
  arXiv  Detail & Related papers  (2025-07-18T17:42:20Z)
• RISE: Robust Imitation through Stochastic Encoding [0.764671395172401]
  We propose a novel imitation-learning framework that explicitly addresses erroneous measurements of environment parameters into policy learning.<n>Our framework encodes parameters such as obstacle state, orientation, and velocity into a latent space to improve test time.<n>We validate our approach on two robotic platforms and demonstrate improved safety while maintaining goal-reaching performance compared to baseline methods.
  arXiv  Detail & Related papers  (2025-03-15T19:52:16Z)
• Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies [19.27526590452503]
  FAIL-Detect is a two-stage approach for failure detection in imitation learning-based robotic manipulation. We first distill policy inputs and outputs into scalar signals that correlate with policy failures and capture uncertainty. Our experiments show learned signals to be mostly consistently effective, particularly when using our novel flow-based density estimator.
  arXiv  Detail & Related papers  (2025-03-11T15:47:12Z)
• STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systems [5.426894918217948]
  STAR (Smart Task Adaptation and Recovery) is a novel framework that synergizes Foundation Models (FMs) with dynamically expanding Knowledge Graphs (KGs) FMs offer remarkable generalization and contextual reasoning, but their limitations hinder reliable deployment. We show that STAR demonstrated an 86% task planning accuracy and 78% recovery success rate, showing significant improvements over baseline methods.
  arXiv  Detail & Related papers  (2025-03-08T05:05:21Z)
• MILE: Model-based Intervention Learning [0.0]
  We show that it is possible to learn a policy with just a handful of expert interventions. Our key insight is that it is possible to get crucial information about the quality of the current state and the optimality of the chosen action from expert feedback.
  arXiv  Detail & Related papers  (2025-02-19T08:15:16Z)
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics [70.93622520400385]
  This paper systematically quantifies the robustness of VLA-based robotic systems. We introduce an untargeted position-aware attack objective that leverages spatial foundations to destabilize robotic actions. We also design an adversarial patch generation approach that places a small, colorful patch within the camera's view, effectively executing the attack in both digital and physical environments.
  arXiv  Detail & Related papers  (2024-11-18T01:52:20Z)
• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• Leveraging Auxiliary Task Relevance for Enhanced Bearing Fault Diagnosis through Curriculum Meta-learning [2.625384087360766]
  A Related Task Aware Curriculum Meta-learning (RT-ACM) enhanced fault diagnosis framework is proposed in this paper. RT-ACM improves training by considering the relevance of auxiliary sensor working conditions. This approach aids the meta-learner in achieving a superior convergence state.
  arXiv  Detail & Related papers  (2024-10-27T06:32:41Z)
• No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery [53.08822154199948]
  Unsupervised Environment Design (UED) methods have gained recent attention as their adaptive curricula promise to enable agents to be robust to in- and out-of-distribution tasks. This work investigates how existing UED methods select training environments, focusing on task prioritisation metrics. We develop a method that directly trains on scenarios with high learnability.
  arXiv  Detail & Related papers  (2024-08-27T14:31:54Z)
• Foundation Models for Autonomous Robots in Unstructured Environments [15.517532442044962]
  The study systematically reviews application of foundation models in two field of robotic and unstructured environment. Findings showed that linguistic capabilities of LLMs have been utilized more than other features for improving perception in human-robot interactions. The use of LLMs demonstrated more applications in project management and safety in construction, and natural hazard detection in disaster management.
  arXiv  Detail & Related papers  (2024-07-19T13:26:52Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• A GP-based Robust Motion Planning Framework for Agile Autonomous Robot Navigation and Recovery in Unknown Environments [6.859965454961918]
  We propose a model for proactively detecting the risk of future motion planning failure. When the risk exceeds a certain threshold, a recovery behavior is triggered. Our framework is capable of both predicting planner failures and recovering the robot to states where planner success is likely.
  arXiv  Detail & Related papers  (2024-02-02T18:27:21Z)
• Bridging Active Exploration and Uncertainty-Aware Deployment Using Probabilistic Ensemble Neural Network Dynamics [11.946807588018595]
  This paper presents a unified model-based reinforcement learning framework that bridges active exploration and uncertainty-aware deployment. The two opposing tasks of exploration and deployment are optimized through state-of-the-art sampling-based MPC. We conduct experiments on both autonomous vehicles and wheeled robots, showing promising results for both exploration and deployment.
  arXiv  Detail & Related papers  (2023-05-20T17:20:12Z)
• Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN [77.0623472106488]
  In this paper, we explore a class of distributional instance segmentation models using latent codes. For robotic picking applications, we propose a confidence mask method to achieve the high precision necessary. We show that our method can significantly reduce critical errors in robotic systems, including our newly released dataset of ambiguous scenes.
  arXiv  Detail & Related papers  (2023-05-03T05:57:29Z)
• Constrained Reinforcement Learning for Robotics via Scenario-Based Programming [64.07167316957533]
  It is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.
  arXiv  Detail & Related papers  (2022-06-20T07:19:38Z)
• Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimization [63.75188254377202]
  Deep reinforcement learning algorithms can perform poorly in real-world tasks due to discrepancy between source and target environments. We propose a novel model-free actor-critic algorithm to learn robust policies without modeling the disturbance in advance. Experiments in several robot control tasks demonstrate that SCPO learns robust policies against the disturbance in transition dynamics.
  arXiv  Detail & Related papers  (2021-12-20T13:13:05Z)
• Error-Aware Policy Learning: Zero-Shot Generalization in Partially Observable Dynamic Environments [18.8481771211768]
  We introduce a novel approach to tackle such a sim-to-real problem by developing policies capable of adapting to new environments. Key to our approach is an error-aware policy (EAP) that is explicitly made aware of the effect of unobservable factors during training. We show that a trained EAP for a hip-torque assistive device can be transferred to different human agents with unseen biomechanical characteristics.
  arXiv  Detail & Related papers  (2021-03-13T15:36:44Z)
• Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms [60.59764170868101]
  Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. We formulate it as a few-shot meta-learning problem where the goal is to find a model that captures the common structure shared across different robotic platforms. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots.
  arXiv  Detail & Related papers  (2021-03-05T14:16:20Z)
• Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning [75.56839075060819]
  Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state. reinforcement learning approaches can operate directly from raw sensory inputs with only a reward signal to describe the task, but are extremely sample-inefficient and brittle. In this work, we combine the strengths of model-based methods with the flexibility of learning-based methods to obtain a general method that is able to overcome inaccuracies in the robotics perception/actuation pipeline.
  arXiv  Detail & Related papers  (2020-05-21T19:47:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.