H2O: A Benchmark for Visual Human-human Object Handover Analysis

• URL: http://arxiv.org/abs/2104.11466v1
• Date: Fri, 23 Apr 2021 08:30:54 GMT
• Title: H2O: A Benchmark for Visual Human-human Object Handover Analysis
• Abstract summary: We propose a novel rich-annotated dataset, H2O, for visual analysis of human-human object handovers. The dataset contains 18K video clips involving 15 people who hand over 30 objects to each other. It can support several vision-based tasks, from which, we specifically provide a baseline method, RGPNet, for a less-explored task named Receiver Grasp Prediction.
• Score: 45.27427828040923
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object handover is a common human collaboration behavior that attracts attention from researchers in Robotics and Cognitive Science. Though visual perception plays an important role in the object handover task, the whole handover process has been specifically explored. In this work, we propose a novel rich-annotated dataset, H2O, for visual analysis of human-human object handovers. The H2O, which contains 18K video clips involving 15 people who hand over 30 objects to each other, is a multi-purpose benchmark. It can support several vision-based tasks, from which, we specifically provide a baseline method, RGPNet, for a less-explored task named Receiver Grasp Prediction. Extensive experiments show that the RGPNet can produce plausible grasps based on the giver's hand-object states in the pre-handover phase. Besides, we also report the hand and object pose errors with existing baselines and show that the dataset can serve as the video demonstrations for robot imitation learning on the handover task. Dataset, model and code will be made public.

Related papers

• Do as I Do: Dexterous Manipulation Data from Everyday Human Videos [69.98851738909168]
  We present DO AS I DO, an algorithm to reconstruct and retarget monocular RGB human videos to multi-fingered dexterous robotic hands.<n>The algorithm then retargets these hand-object interaction estimates into a sequence of actions executable in the real world.<n>Overall, DO AS I DO outperforms previous state of the art in estimating hand-object interactions and extracting dexterous manipulation trajectories from RGB videos.
  arXiv  Detail & Related papers  (2026-06-17T17:57:34Z)
• SIGHT: Single-Image Conditioned Generation of Hand Trajectories for Hand-Object Interaction [86.54738165527502]
  We introduce a novel task of generating realistic and diverse 3D hand trajectories given a single image of an object. Hand-object interaction trajectory priors can greatly benefit applications in robotics, embodied AI, augmented reality and related fields.
  arXiv  Detail & Related papers  (2025-03-28T20:53:20Z)
• G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis [57.07638884476174]
  G-HOP is a denoising diffusion based generative prior for hand-object interactions. We represent the human hand via a skeletal distance field to obtain a representation aligned with the signed distance field for the object. We show that this hand-object prior can then serve as generic guidance to facilitate other tasks like reconstruction from interaction clip and human grasp synthesis.
  arXiv  Detail & Related papers  (2024-04-18T17:59:28Z)
• HOH: Markerless Multimodal Human-Object-Human Handover Dataset with Large Object Count [4.441651788685931]
  The HOH (Human-Object-Human) Handover dataset is a large object count dataset with 136 objects. HOH represents natural human-human handover interactions, overcoming challenges with markered datasets. To date, HOH is the largest handover dataset in number of objects, participants, pairs with role reversal accounted for, and total interactions captured.
  arXiv  Detail & Related papers  (2023-10-01T16:48:48Z)
• AffordPose: A Large-scale Dataset of Hand-Object Interactions with Affordance-driven Hand Pose [16.65196181081623]
  We present AffordPose, a large-scale dataset of hand-object interactions with affordance-driven hand pose. We collect a total of 26.7K hand-object interactions, each including the 3D object shape, the part-level affordance label, and the manually adjusted hand poses. The comprehensive data analysis shows the common characteristics and diversity of hand-object interactions per affordance.
  arXiv  Detail & Related papers  (2023-09-16T10:25:28Z)
• Human-Object Interaction Prediction in Videos through Gaze Following [9.61701724661823]
  We design a framework to detect current HOIs and anticipate future HOIs in videos. We propose to leverage human information since people often fixate on an object before interacting with it. Our model is trained and validated on the VidHOI dataset, which contains videos capturing daily life.
  arXiv  Detail & Related papers  (2023-06-06T11:36:14Z)
• Learn to Predict How Humans Manipulate Large-sized Objects from Interactive Motions [82.90906153293585]
  We propose a graph neural network, HO-GCN, to fuse motion data and dynamic descriptors for the prediction task. We show the proposed network that consumes dynamic descriptors can achieve state-of-the-art prediction results and help the network better generalize to unseen objects.
  arXiv  Detail & Related papers  (2022-06-25T09:55:39Z)
• REGRAD: A Large-Scale Relational Grasp Dataset for Safe and Object-Specific Robotic Grasping in Clutter [52.117388513480435]
  We present a new dataset named regrad to sustain the modeling of relationships among objects and grasps. Our dataset is collected in both forms of 2D images and 3D point clouds. Users are free to import their own object models for the generation of as many data as they want.
  arXiv  Detail & Related papers  (2021-04-29T05:31:21Z)
• Few-Shot Visual Grounding for Natural Human-Robot Interaction [0.0]
  We propose a software architecture that segments a target object from a crowded scene, indicated verbally by a human user. At the core of our system, we employ a multi-modal deep neural network for visual grounding. We evaluate the performance of the proposed model on real RGB-D data collected from public scene datasets.
  arXiv  Detail & Related papers  (2021-03-17T15:24:02Z)
• Reactive Human-to-Robot Handovers of Arbitrary Objects [57.845894608577495]
  We present a vision-based system that enables human-to-robot handovers of unknown objects. Our approach combines closed-loop motion planning with real-time, temporally-consistent grasp generation. We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects.
  arXiv  Detail & Related papers  (2020-11-17T21:52:22Z)
• Human Grasp Classification for Reactive Human-to-Robot Handovers [50.91803283297065]
  We propose an approach for human-to-robot handovers in which the robot meets the human halfway. We collect a human grasp dataset which covers typical ways of holding objects with various hand shapes and poses. We present a planning and execution approach that takes the object from the human hand according to the detected grasp and hand position.
  arXiv  Detail & Related papers  (2020-03-12T19:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.