R2HandoverSim: A Simulation Framework and Benchmark for Robot-to-Human Object Handovers

• URL: http://arxiv.org/abs/2606.21011v1
• Date: Fri, 19 Jun 2026 00:48:06 GMT
• Title: R2HandoverSim: A Simulation Framework and Benchmark for Robot-to-Human Object Handovers
• Abstract summary: R2HandoverSim is a simulation benchmark for robot-to-human (R2H) object handovers.<n>Our benchmark enables reproducible evaluation by comparing four baselines on their predicted shared grasp poses.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present R2HandoverSim, a simulation benchmark for robot-to-human (R2H) object handovers. Although R2H handover methods have advanced rapidly, the lack of standardized evaluation protocols impedes objective comparison. Our benchmark enables reproducible evaluation by systematically comparing four baselines on their predicted shared grasp poses. We conduct a user study with 30 participants, analyze baseline performance, and show that simulation results correlate with real-world evaluation outcomes. Crucially, five complementary metrics (planning feasibility, reachability, grasp stability, grasp affordance, and safety) better reflect user-perceived handover quality than overall success rate alone. Website and code: https://robot-future.github.io/r2handoversim/.

Related papers

• R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models [12.45799943639499]
  R2S-Eval is an evaluation pipeline that combines real-to-sim calibration with vision-language model (VLM) preference evaluation.<n>It produces reliable and stable policy conclusions, achieves agreement with human preferences, substantially reduces repeated hardware-operation effort, and reveals behavior-quality differences that are not captured by binary success labels.
  arXiv  Detail & Related papers  (2026-09-03T02:08:16Z)
• RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction [51.76737991200891]
  Human-to-robot (H2R) object handover is a fundamental capability for human-robot collaboration, yet progress is hindered by the scarcity of large-scale, human-centric datasets.<n>We introduce Hand2Bot, an RGB-D video dataset that provides rich contextual information such as body posture and facial expressions.<n>We propose PassGen, a generative pipeline that synthesizes realistic handover sequences while ensuring hand-object consistency.
  arXiv  Detail & Related papers  (2026-08-13T09:55:59Z)
• RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies [121.35821705538768]
  Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities.<n>We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies.<n>RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities.
  arXiv  Detail & Related papers  (2026-07-05T17:58:02Z)
• Toward Visually Realistic Simulation: A Benchmark for Evaluating Robot Manipulation in Simulation [62.51953630639423]
  Existing benchmarks lack visual realism, creating a large domain gap between simulation and reality.<n>We propose VISER, a visually realistic benchmark for evaluating robot manipulation in simulation.<n>VISER features a high-fidelity dataset of over 1,000 3D assets with physically-based rendering (PBR) materials, along with 3D scenes created from these assets through curated layouts or generation.
  arXiv  Detail & Related papers  (2026-05-07T14:13:05Z)
• SimEval-IR: A Unified Toolkit and Benchmark Suite for Evaluating User Simulators and Search Sessions [1.1105673928718571]
  We present SimEval-IR, an open-source toolkit and benchmark suite that makes the distinction measurable.<n>SimEval-IR provides: (1) a canonical session schema unifying session search and conversational interactions, with validated dataset adapters and explicit loss accounting; (2) three executable benchmarks covering behavioral realism, tester reliability with RATE-style estimation, and an analysis linking the two; and (3) baseline results across four real datasets in two languages and four simulator families.
  arXiv  Detail & Related papers  (2026-04-30T13:56:18Z)
• Mind the Sim2Real Gap in User Simulation for Agentic Tasks [101.69142591891234]
  We formalize the Sim2Real gap in user simulation and present the first study running the full $$-bench protocol with real humans.<n> Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity.<n>These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle.
  arXiv  Detail & Related papers  (2026-03-11T19:12:31Z)
• PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies [88.78188489161028]
  We introduce Policy Evaluation and Environment Reconstruction in Simulation (PolaRiS)<n>PolaRiS is a scalable real-to-sim framework for high-fidelity simulated robot evaluation.<n>We show that PolaRiS evaluations provide a much stronger correlation to real world generalist policy performance than existing simulated benchmarks.
  arXiv  Detail & Related papers  (2025-12-18T18:49:41Z)
• SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? [61.07963107032645]
  Large language models (LLMs) are increasingly used in interactive applications.<n>Human evaluation remains the gold standard for assessing their performance in multi-turn conversations.<n>We introduce SimulatorArena, a benchmark of 909 annotated human-LLM conversations on two interactive tasks.
  arXiv  Detail & Related papers  (2025-10-06T23:17:44Z)
• Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators [9.868826622074899]
  SureSim is a framework to augment large-scale simulation with relatively small-scale real-world testing.<n>We leverage non-asymptotic mean estimation algorithms to provide confidence intervals on mean policy performance.<n>Our approach saves over (20-25%) of hardware evaluation effort to achieve similar bounds on policy performance.
  arXiv  Detail & Related papers  (2025-10-05T20:37:53Z)
• Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction [26.765285050287247]
  We propose a general evaluation framework that measures the quality of Imitation Learning (IL) methods by focusing on trajectory performance.<n>We devise the Neural Meta Evaluator (NeME), a deep learning model trained to classify actions from robot joint trajectories.<n>We validate our framework on ergoCub, a humanoid robot, using teleoperation data and comparing IL methods tailored to the available platform.
  arXiv  Detail & Related papers  (2025-07-08T21:12:57Z)
• Pseudo-Simulation for Autonomous Driving [66.1981253104508]
  Existing evaluation paradigms for Autonomous Vehicles (AVs) face critical limitations.<n>Real-world evaluation is often challenging due to safety concerns and a lack of realism.<n>Open-loop evaluation relies on metrics that generally overlook compounding errors.
  arXiv  Detail & Related papers  (2025-06-04T17:57:53Z)
• HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers [60.45158007016316]
  "HandoverSim" is a simulation benchmark for human-to-robot object handovers. We leverage a recent motion capture dataset of hand grasping of objects. We create training and evaluation environments for the receiver with standardized protocols and metrics.
  arXiv  Detail & Related papers  (2022-05-19T17:59:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.