REPAIR-Bench: A Benchmark for Robot Error Perception And Interaction Recovery

• URL: http://arxiv.org/abs/2606.29937v1
• Date: Mon, 29 Jun 2026 08:14:27 GMT
• Title: REPAIR-Bench: A Benchmark for Robot Error Perception And Interaction Recovery
• Abstract summary: We present-Bench, built on 214 interaction trials from 41 participants.<n>The benchmark spans four induced failure types and provides synchronized facial action units, head pose, speech transcripts, and post-interaction affect and recovery reports.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how users perceive and respond to robot failures is essential for building robust and trustworthy robot systems. Prior work, however, (i) often treats failures as independent events, (ii) emphasizes binary failure detection, (iii) with rule-based recovery modeling. We present REPAIR-Bench, built on 214 interaction trials from 41 participants, the benchmark spans four induced failure types and provides synchronized facial action units, head pose, speech transcripts, and post-interaction affect and recovery reports. The benchmark spans three novel evaluation tasks that jointly capture the lifecycle of failure in human-robot interaction (HRI): (i) failure detection over inter-dependent interaction sessions, modeling longitudinal user adaptation across repeated failures; (ii) visual failure-type classification beyond binary success/failure formulations; and (iii) user-centered recovery prediction, inferring users' preferred recovery strategies from interaction context rather than relying on manually designed or rule-based strategies. In baseline experiments, hierarchical recurrent modeling improved failure detection over a single-session model (strict F1: 0.80 vs. 0.68), achieved a failure localization mean signed error of -0.51 s, median absolute error of 2.97 s and, for recovery prediction, a QLoRA-tuned Mistral-7B reached Hit@5=0.76 and F1@5=0.32. REPAIR-Bench provides both the HRI and Medical HRI communities with a standardized framework for (1) evaluating robot failures and (2) building transparent, adaptive, and trustworthy recovery systems.

Related papers

• FailBench: How Reliable are VLMs at Judging Robot Task Success? [0.0]
  FailBench is a benchmark for robot failure detection comprising 2,197 manipulation attempts across 14 public sources.<n>In FailBench, 75% of failures occur naturally, and six real-world sources come from non-failure-detection datasets.
  arXiv  Detail & Related papers  (2026-09-03T09:58:55Z)
• Autonomous Telerehabilitation via Skeletal Motion Prediction and Joint-Level Performance Assessment [0.8283026597815732]
  This paper presents a telerehabilitation pipeline that integrates skeleton-based exercise quality assessment and short-term motion prediction.<n>A self-attentive Bidirectional LSTM performs exercise quality classification using MMD-NCA metric learning.<n>A graph-based motion prediction module computes per-joint position errors between predicted and observed poses, generating spatially localized deviation signals.
  arXiv  Detail & Related papers  (2026-08-12T15:06:51Z)
• ERR@HRI 3.0 Challenge: Multimodal Detection of Errors and Anticipation in Human-Robot Interactions [18.151307410451796]
  The third edition of the ERR@HRI Challenge (ERR@HRI 3.0) provided researchers with two complementary datasets.<n>This paper describes the datasets, tasks, baselines, and results of ERR@HRI 3.0, and discusses implications for building generalizable, context-aware, and anticipatory error detection systems.
  arXiv  Detail & Related papers  (2026-07-13T13:50:43Z)
• SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation [73.2142090645987]
  SC3-Eval is a self-consistent video generation recipe that adapts a pre-trained video foundation model into an accurate policy evaluator.<n>We demonstrate SC3-Eval rollouts reproduce the failure modes that policies exhibit in real-world rollouts, supporting fine-grained diagnostic comparison.
  arXiv  Detail & Related papers  (2026-06-17T02:15:46Z)
• FailureScope: Cross-Regime Behavioral Diagnosis of Language Model Weaknesses [0.0]
  FailureScope is a behavioral-diagnosis method that clusters evaluation probes by their cross-model pass/fail patterns.<n>We show it yields stable, interpretable failure across three regimes usually studied separately: single-turn benchmarks, multi-turn dialogue, and adversarial agent attacks.
  arXiv  Detail & Related papers  (2026-06-03T01:28:00Z)
• When Web Apps Heal Themselves: A MAPE-K Based Approach to Fault Tolerance and Adaptive Recovery [0.0]
  This study proposes a modular self-healing framework based on the monitor-analyze-plan-execute over a shared knowledge base.<n>It is integrated with an AutoFix-inspired mechanism for adaptive fault recovery.<n>The proposed framework achieved a mean fault detection F1-score of 90.7% and a recovery success rate of 93.2%.
  arXiv  Detail & Related papers  (2026-05-19T02:17:24Z)
• Doubly Robust Proxy Causal Learning with Neural Mean Embeddings [58.8808794752741]
  We develop a neural doubly robust framework for proxy causal learning with continuous and structured treatments.<n>The framework covers population, heterogeneous, and conditional dose-response functions, yielding full response-curve estimators.<n>Across synthetic and image-valued benchmarks, the proposed estimators outperform existing baselines and single-bridge neural estimators.
  arXiv  Detail & Related papers  (2026-05-10T12:50:25Z)
• ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models [60.14219417402433]
  Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.<n>We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three constructs drawn from standard cognitive-science accounts of non-declarative memory.<n>Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring.
  arXiv  Detail & Related papers  (2026-04-09T10:26:32Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• Learning Actionable Manipulation Recovery via Counterfactual Failure Synthesis [21.197844940385725]
  Current failure-learning paradigms rely on either costly and unsafe real-world data collection or simulator-based perturbations.<n>We introduce Dream2Fix, a framework that synthesizes photorealistic, counterfactual failure rollouts directly from successful real-world demonstrations.<n>By perturbing actions within a generative world model, Dream2Fix creates paired failure-language data without relying on simulators.
  arXiv  Detail & Related papers  (2026-03-13T19:02:58Z)
• RFM-HRI : A Multimodal Dataset of Medical Robot Failure, User Reaction and Recovery Preferences for Item Retrieval Tasks [0.0]
  This work contributes (1) a publicly available multimodal dataset (RFM-HRI), (2) analysis of user responses to different failure types and preferred recovery strategies, and (3) a crash-cart retrieval scenario enabling systematic comparison of recovery strategies with implications for safety-critical failure recovery.
  arXiv  Detail & Related papers  (2026-03-05T19:52:19Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Recursive Knowledge Synthesis for Multi-LLM Systems: Stability Analysis and Tri-Agent Audit Framework [0.0]
  This paper presents a tri-agent cross-validation framework for analyzing stability and explainability in large language systems.<n>The architecture integrates three heterogeneous LLMs used for semantic generation, analytical consistency checking, and transparency auditing.
  arXiv  Detail & Related papers  (2025-12-17T16:42:45Z)
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis [55.13545823385091]
  Federated reinforcement learning (FedRL) enables collaborative learning while preserving data privacy by preventing direct data exchange between agents.<n>In real-world applications, each agent may experience slightly different transition dynamics, leading to inherent model mismatches.<n>We show that even moderate levels of information sharing significantly mitigate environment-specific errors.
  arXiv  Detail & Related papers  (2025-03-21T18:06:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.