Hand-Object Contact Consistency Reasoning for Human Grasps Generation
- URL: http://arxiv.org/abs/2104.03304v1
- Date: Wed, 7 Apr 2021 17:57:14 GMT
- Title: Hand-Object Contact Consistency Reasoning for Human Grasps Generation
- Authors: Hanwen Jiang, Shaowei Liu, Jiashun Wang and Xiaolong Wang
- Abstract summary: We propose to generate human grasps given a 3D object in the world.
Key observation is that it is crucial to model the consistency between the hand contact points and object contact regions.
Experiments show significant improvement in human grasp generation over state-of-the-art approaches by a large margin.
- Score: 6.398433415259542
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: While predicting robot grasps with parallel jaw grippers have been well
studied and widely applied in robot manipulation tasks, the study on natural
human grasp generation with a multi-finger hand remains a very challenging
problem. In this paper, we propose to generate human grasps given a 3D object
in the world. Our key observation is that it is crucial to model the
consistency between the hand contact points and object contact regions. That
is, we encourage the prior hand contact points to be close to the object
surface and the object common contact regions to be touched by the hand at the
same time. Based on the hand-object contact consistency, we design novel
objectives in training the human grasp generation model and also a new
self-supervised task which allows the grasp generation network to be adjusted
even during test time. Our experiments show significant improvement in human
grasp generation over state-of-the-art approaches by a large margin. More
interestingly, by optimizing the model during test time with the
self-supervised task, it helps achieve larger gain on unseen and out-of-domain
objects. Project page: https://hwjiang1510.github.io/GraspTTA/
Related papers
- Target Pose Guided Whole-body Grasping Motion Generation for Digital Humans [8.741075482543991]
We propose a grasping motion generation framework for digital human.
We first generate a target pose for whole-body digital human based on off-the-shelf target grasping pose generation methods.
With an initial pose and this generated target pose, a transformer-based neural network is used to generate the whole grasping trajectory.
arXiv Detail & Related papers (2024-09-26T05:43:23Z) - ContactHandover: Contact-Guided Robot-to-Human Object Handover [23.093164853009547]
We propose a robot to human handover system that consists of two phases: a contact-guided grasping phase and an object delivery phase.
During the grasping phase, ContactHandover predicts both 6-DoF robot grasp poses and a 3D affordance map of human contact points on the object.
During the delivery phase, the robot end effector pose is computed by maximizing human contact points close to the human while minimizing the human arm joint torques and displacements.
arXiv Detail & Related papers (2024-04-01T18:12:09Z) - Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction [8.253265795150401]
This paper introduces the first text-guided work for generating the sequence of hand-object interaction in 3D.
For contact generation, a VAE-based network takes as input a text and an object mesh, and generates the probability of contacts between the surfaces of hands and the object.
For motion generation, a Transformer-based diffusion model utilizes this 3D contact map as a strong prior for generating physically plausible hand-object motion.
arXiv Detail & Related papers (2024-03-31T04:56:30Z) - Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects [89.95728475983263]
holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation.
We design the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits.
Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks.
arXiv Detail & Related papers (2024-03-25T05:12:21Z) - GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistency [57.9920824261925]
Hands are dexterous and highly versatile manipulators that are central to how humans interact with objects and their environment.
modeling realistic hand-object interactions is critical for applications in computer graphics, computer vision, and mixed reality.
GRIP is a learning-based method that takes as input the 3D motion of the body and the object, and synthesizes realistic motion for both hands before, during, and after object interaction.
arXiv Detail & Related papers (2023-08-22T17:59:51Z) - Task-Oriented Human-Object Interactions Generation with Implicit Neural
Representations [61.659439423703155]
TOHO: Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations.
Our method generates continuous motions that are parameterized only by the temporal coordinate.
This work takes a step further toward general human-scene interaction simulation.
arXiv Detail & Related papers (2023-03-23T09:31:56Z) - Contact-aware Human Motion Forecasting [87.04827994793823]
We tackle the task of scene-aware 3D human motion forecasting, which consists of predicting future human poses given a 3D scene and a past human motion.
Our approach outperforms the state-of-the-art human motion forecasting and human synthesis methods on both synthetic and real datasets.
arXiv Detail & Related papers (2022-10-08T07:53:19Z) - Model Predictive Control for Fluid Human-to-Robot Handovers [50.72520769938633]
Planning motions that take human comfort into account is not a part of the human-robot handover process.
We propose to generate smooth motions via an efficient model-predictive control framework.
We conduct human-to-robot handover experiments on a diverse set of objects with several users.
arXiv Detail & Related papers (2022-03-31T23:08:20Z) - Grasping Field: Learning Implicit Representations for Human Grasps [16.841780141055505]
We propose an expressive representation for human grasp modelling that is efficient and easy to integrate with deep neural networks.
We name this 3D to 2D mapping as Grasping Field, parameterize it with a deep neural network, and learn it from data.
Our generative model is able to synthesize high-quality human grasps, given only on a 3D object point cloud.
arXiv Detail & Related papers (2020-08-10T23:08:26Z) - Human Grasp Classification for Reactive Human-to-Robot Handovers [50.91803283297065]
We propose an approach for human-to-robot handovers in which the robot meets the human halfway.
We collect a human grasp dataset which covers typical ways of holding objects with various hand shapes and poses.
We present a planning and execution approach that takes the object from the human hand according to the detected grasp and hand position.
arXiv Detail & Related papers (2020-03-12T19:58:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.