Why did I fail? A Causal-based Method to Find Explanations for Robot Failures

• URL: http://arxiv.org/abs/2204.04483v1
• Date: Sat, 9 Apr 2022 14:35:43 GMT
• Title: Why did I fail? A Causal-based Method to Find Explanations for Robot Failures
• Abstract summary: The ability of robots to explain such failures is paramount for interacting with humans to increase trust and transparency. To achieve this skill, the main challenges addressed in this paper are I) acquiring enough data to learn a cause-effect model of the environment and II) generating causal explanations based on that model. We propose a novel method that enables robots to generate contrastive explanations upon task failures.
• Score: 0.17132914341329847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robot failures in human-centered environments are inevitable. Therefore, the ability of robots to explain such failures is paramount for interacting with humans to increase trust and transparency. To achieve this skill, the main challenges addressed in this paper are I) acquiring enough data to learn a cause-effect model of the environment and II) generating causal explanations based on that model. We address I) by learning a causal Bayesian network from simulation data. Concerning II), we propose a novel method that enables robots to generate contrastive explanations upon task failures. The explanation is based on setting the failure state in contrast with the closest state that would have allowed for successful execution, which is found through breadth-first search and is based on success predictions from the learned causal model. We assess the sim2real transferability of the causal model on a cube stacking scenario. Based on real-world experiments with two differently embodied robots, we achieve a sim2real accuracy of 70% without any adaptation or retraining. Our method thus allowed real robots to give failure explanations like, 'the upper cube was dropped too high and too far to the right of the lower cube.'

Related papers

• Robot Self-Improvement via Human-Video Dynamics Models [52.14862052988773]
  We show that human videos can be used to learn embodiment-agnostic action, dynamics, and value representations that transfer across robot embodiments.<n>We introduce Dynamics-Guided Action Correction (DGAC), a training-free approach that leverages these adapted models to repair failed states.<n>Results show that human priors and robot failures can be combined to enable scalable autonomous policy improvement.
  arXiv  Detail & Related papers  (2026-06-19T13:17:27Z)
• HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining [93.5192770515694]
  Embodied foundation models are expected to benefit from data scaling like large language models, but face a much tighter data bottleneck.<n>Teleoperated real-robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment alignment.<n>Embodied foundation models pretrained on egocentric data achieve a 24% lower validation loss on real-robot action prediction.
  arXiv  Detail & Related papers  (2026-06-18T17:37:34Z)
• OASIS: From Simulation Data Collection to Real-World Humanoid Loco-Manipulation [51.37436817948304]
  OASIS is a simulation-data-driven framework for humanoid loco-manipulation.<n>It reconstructs realistic object assets from real-world images using a 3D generative model.<n>Under zero-shot deployment, it achieves higher success rates on most tasks than that trained on real-robot teleoperation data.
  arXiv  Detail & Related papers  (2026-06-07T10:01:53Z)
• A Causal Approach to Predicting and Improving Human Perceptions of Social Navigation Robots [0.9134244356393665]
  Causal Network designed to predict how people perceive a mobile robot's competence and how they interpret its intent during navigation.<n>Method statistically significantly increased perceived competence of low-competent robot behavior by 83%.
  arXiv  Detail & Related papers  (2026-03-11T20:39:11Z)
• The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity? [53.15349353876531]
  As AI becomes more capable, we entrust it with more general and consequential tasks.<n>We operationalize this question using a bias-variance decomposition of the errors made by AI models.<n>As more capable AIs pursue harder tasks, requiring more sequential action and thought, our results predict failures to be accompanied by more incoherent behavior.
  arXiv  Detail & Related papers  (2026-01-30T14:52:03Z)
• Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models [53.20969621498248]
  We propose an automatic robot failure synthesis approach that procedurally perturbs successful trajectories to generate diverse planning and execution failures.<n>We construct three new failure detection benchmarks: RLBench-Fail, BridgeDataV2-Fail, and UR5-Fail.<n>We then train Guardian, a VLM with multi-view images for detailed failure reasoning and detection.
  arXiv  Detail & Related papers  (2025-12-01T17:57:27Z)
• Action Flow Matching for Continual Robot Learning [57.698553219660376]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks. We introduce a generative framework leveraging flow matching for online robot dynamics model alignment. We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• Offline Imitation Learning Through Graph Search and Retrieval [57.57306578140857]
  Imitation learning is a powerful machine learning algorithm for a robot to acquire manipulation skills. We propose GSR, a simple yet effective algorithm that learns from suboptimal demonstrations through Graph Search and Retrieval. GSR can achieve a 10% to 30% higher success rate and over 30% higher proficiency compared to baselines.
  arXiv  Detail & Related papers  (2024-07-22T06:12:21Z)
• On-Robot Bayesian Reinforcement Learning for POMDPs [16.667924736270415]
  This paper advances Bayesian reinforcement learning for robotics by proposing a specialized framework for physical systems. We capture this knowledge in a factored representation, then demonstrate the posterior factorizes in a similar shape, and ultimately formalize the model in a Bayesian framework. We then introduce a sample-based online solution method, based on Monte-Carlo tree search and particle filtering, specialized to solve the resulting model.
  arXiv  Detail & Related papers  (2023-07-22T01:16:29Z)
• Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN [77.0623472106488]
  In this paper, we explore a class of distributional instance segmentation models using latent codes. For robotic picking applications, we propose a confidence mask method to achieve the high precision necessary. We show that our method can significantly reduce critical errors in robotic systems, including our newly released dataset of ambiguous scenes.
  arXiv  Detail & Related papers  (2023-05-03T05:57:29Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• Scaling Robot Learning with Semantically Imagined Experience [21.361979238427722]
  Recent advances in robot learning have shown promise in enabling robots to perform manipulation tasks. One of the key contributing factors to this progress is the scale of robot data used to train the models. We propose an alternative route and leverage text-to-image foundation models widely used in computer vision and natural language processing.
  arXiv  Detail & Related papers  (2023-02-22T18:47:51Z)
• Optimal decision making in robotic assembly and other trial-and-error tasks [1.0660480034605238]
  We study a class of problems providing (1) low-entropy indicators of terminal success / failure, and (2) unreliable (high-entropy) data to predict the final outcome of an ongoing task. We derive a closed form solution that predicts makespan based on the confusion matrix of the failure predictor. This allows the robot to learn failure prediction in a production environment, and only adopt a preemptive policy when it actually saves time.
  arXiv  Detail & Related papers  (2023-01-25T22:07:50Z)
• Robot Learning from Randomized Simulations: A Review [59.992761565399185]
  Deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. State-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive. We focus on a technique named 'domain randomization' which is a method for learning from randomized simulations.
  arXiv  Detail & Related papers  (2021-11-01T13:55:41Z)
• SIM2REALVIZ: Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation [8.272193665645553]
  We introduce Sim2RealViz, a visual analytics tool to assist experts in understanding and reducing this gap for robot ego-pose estimation tasks. Sim2RealViz displays details of a given model and the performance of its instances in both simulation and real-world.
  arXiv  Detail & Related papers  (2021-09-24T08:17:33Z)
• Reactive Long Horizon Task Execution via Visual Skill and Precondition Models [59.76233967614774]
  We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a simple task planner. We show an increase in success rate from 91.6% to 98% in simulation and from 10% to 80% success rate in the real-world as compared with naive baselines.
  arXiv  Detail & Related papers  (2020-11-17T15:24:01Z)
• Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs [90.20235972293801]
  Aiming to understand how human (false-temporal)-belief-a core socio-cognitive ability unify-would affect human interactions with robots, this paper proposes to adopt a graphical model to the representation of object states, robot knowledge, and human (false-)beliefs. An inference algorithm is derived to fuse individual pg from all robots across multi-views into a joint pg, which affords more effective reasoning inference capability to overcome the errors originated from a single view.
  arXiv  Detail & Related papers  (2020-04-25T23:02:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.