How Visible Are Silent Manipulation Failures? An Observability Study of False-Success Detection in Simulated Robot Episodes

• URL: http://arxiv.org/abs/2606.03134v1
• Date: Tue, 02 Jun 2026 04:22:13 GMT
• Title: How Visible Are Silent Manipulation Failures? An Observability Study of False-Success Detection in Simulated Robot Episodes
• Abstract summary: Imitation-learning policies for robot manipulation inherit the quality of the success labels attached to their training episodes.<n>A particularly damaging error is the false success: an episode the robot logs as a success when the task outcome was actually wrong.<n>We build a simulated testbed on two bimanual ALOHA tasks, induce failures through environment perturbations rather than label edits, label every episode by privileged simulator state that the detector never sees, and keep only episodes the robot flagged as successful.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation-learning policies for robot manipulation inherit the quality of the success labels attached to their training episodes, and those labels are usually produced by the robot's own success check. A particularly damaging error is the false success: an episode the robot logs as a success when the task outcome was actually wrong. We ask a narrow but practical question about these episodes. Once an episode has already been flagged as a success, how much of the information needed to overturn that label is present in proprioception, and how much requires vision? We build a simulated testbed on two bimanual ALOHA tasks, induce failures through environment perturbations rather than label edits, label every episode by privileged simulator state that the detector never sees, and keep only episodes the robot flagged as successful. We then compare detectors restricted to proprioception against a vision-based detector. We find that recoverability spans a wide range: in cube transfer the false successes are almost fully recoverable from joint data alone, while in peg insertion proprioception recovers only part of them and a vision detector closes most of the gap. We also show that the proprioceptive separability we measure rests on velocity differences far below any realistic sensor noise floor, so it is best read as an optimistic upper bound that a noiseless simulator inflates. We release the generation and evaluation pipeline.

Related papers

• DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulation [67.08835970838996]
  Reinforcement learning holds great promise for improving robot policies beyond the limits of imitation learning.<n>Two key challenges remain: acquiring diverse failure data at scale and obtaining fine-grained reward signals beyond sparse trajectory-level success labels.<n>We introduce DenseReward, a dense robotic reward model that addresses both challenges.
  arXiv  Detail & Related papers  (2026-07-14T17:59:29Z)
• DemoBridge: A Simulation-in-the-Loop Toolkit for Single-View Human Demonstration Retargeting [49.70827726228219]
  We present DemoBridge, a toolkit that turns a single-view RGB stereo recording of a human hand demonstration into an executable, physics-validated robot-arm trajectory.<n>At the core of DemoBridge is a single collision-aware planner.<n>A physics simulator runs in the loop. It validates each phase as it is produced and backtracks on failure, so a demonstration that cannot be reproduced as given is re-planned rather than discarded.
  arXiv  Detail & Related papers  (2026-07-10T15:31:04Z)
• Foresight: Failure Detection for Long-Horizon Robotic Manipulation with Action-Conditioned World Model Latents [58.49879338545782]
  Long-horizon tasks are common in real-world robotic deployments, yet failure detection for such tasks remains underexplored.<n>We present Foresight, a failure detection framework that monitors manipulation trajectories using latent representations from an action-conditioned world model.<n>Our results suggest that action-conditioned world-model embeddings provide a scalable representation for reliable failure monitoring in long-horizon manipulation.
  arXiv  Detail & Related papers  (2026-06-22T09:32:28Z)
• What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos? [51.606902362959495]
  We use a new dataset of 532 human videos with 28 hours of high-quality triangulated hand labels and natural motions.<n>We find that hand pose quality affects transfer, but even with accurate hands, the inherent motion gap hinders transfer unless the vision and policy networks specialize to each embodiment.
  arXiv  Detail & Related papers  (2026-06-04T18:24:23Z)
• Impact of Different Failures on a Robot's Perceived Reliability [9.607560620828718]
  We examine how various kinds of failures affect the robot's perceived reliability (PR)<n>We show that successes immediately following a failure have the same effect on PR as successes without a preceding failure.<n>Our findings highlight which robot failures are in higher need of repair in a human-robot interaction, and how trust could be recovered by robot successes.
  arXiv  Detail & Related papers  (2026-03-09T18:27:53Z)
• RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learning [47.25770917635344]
  We introduce RoboCurate, a novel synthetic robot data generation framework that evaluates and filters the quality of annotated actions.<n>Specifically, RoboCurate replays the predicted actions in a simulator and assesses action quality by measuring the consistency of motion.<n>We observe RoboCurate's generated data yield substantial relative improvements in success rates compared to using real data only.
  arXiv  Detail & Related papers  (2026-02-21T07:33:24Z)
• Guardian: Detecting Robotic Planning and Execution Errors with Vision-Language Models [53.20969621498248]
  We propose an automatic robot failure synthesis approach that procedurally perturbs successful trajectories to generate diverse planning and execution failures.<n>We construct three new failure detection benchmarks: RLBench-Fail, BridgeDataV2-Fail, and UR5-Fail.<n>We then train Guardian, a VLM with multi-view images for detailed failure reasoning and detection.
  arXiv  Detail & Related papers  (2025-12-01T17:57:27Z)
• When a Robot is More Capable than a Human: Learning from Constrained Demonstrators [4.015444385806047]
  Learning from demonstrations enables experts to teach robots complex tasks using interfaces such as kinesthetic teaching, joystick control, and sim-to-real transfer.<n>These interfaces often constrain the expert's ability to demonstrate optimal behavior due to indirect control, setup restrictions, and hardware safety.<n>This raises a key question: Can a robot learn a better policy than the one demonstrated by a constrained expert?<n>We address this by allowing the agent to go beyond direct imitation of expert actions and explore shorter and more efficient trajectories.
  arXiv  Detail & Related papers  (2025-10-10T07:48:12Z)
• ILeSiA: Interactive Learning of Robot Situational Awareness from Camera Input [13.374743857520487]
  This paper focuses on teaching the robot situational awareness by using a camera input and labeling frames as safe or risky.<n>The proposed method can reliably detect both known and novel faults using only a single example for each new fault.<n>Our method enables the next generation of cobots to be rapidly deployed with easy-to-set-up, vision-based risk assessment.
  arXiv  Detail & Related papers  (2024-09-30T10:31:27Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• Examining Audio Communication Mechanisms for Supervising Fleets of Agricultural Robots [2.76240219662896]
  We develop a simulation platform where agbots are deployed across a field, randomly encounter failures, and call for help from the operator. As the agbots report errors, various audio communication mechanisms are tested to convey which robot failed and what type of failure occurs. A user study was conducted to test three audio communication methods: earcons, single-phrase commands, and full sentence communication.
  arXiv  Detail & Related papers  (2022-08-22T17:19:20Z)
• Why did I fail? A Causal-based Method to Find Explanations for Robot Failures [0.17132914341329847]
  The ability of robots to explain such failures is paramount for interacting with humans to increase trust and transparency. To achieve this skill, the main challenges addressed in this paper are I) acquiring enough data to learn a cause-effect model of the environment and II) generating causal explanations based on that model. We propose a novel method that enables robots to generate contrastive explanations upon task failures.
  arXiv  Detail & Related papers  (2022-04-09T14:35:43Z)
• Where is my hand? Deep hand segmentation for visual self-recognition in humanoid robots [129.46920552019247]
  We propose the use of a Convolution Neural Network (CNN) to segment the robot hand from an image in an egocentric view. We fine-tuned the Mask-RCNN network for the specific task of segmenting the hand of the humanoid robot Vizzy.
  arXiv  Detail & Related papers  (2021-02-09T10:34:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.