Anthropomorphic Grasping with Neural Object Shape Completion
- URL: http://arxiv.org/abs/2311.02510v2
- Date: Thu, 9 Nov 2023 15:06:38 GMT
- Title: Anthropomorphic Grasping with Neural Object Shape Completion
- Authors: Diego Hidalgo-Carvajal, Hanzhi Chen, Gemma C. Bettelani, Jaesug Jung,
Melissa Zavaglia, Laura Busse, Abdeldjallil Naceri, Stefan Leutenegger, Sami
Haddadin
- Abstract summary: Humans exhibit extraordinary dexterity when handling objects.
Hand postures commonly demonstrate the influence of specific regions on objects that need to be grasped.
In this work, we leverage human-like object understanding by reconstructing and completing their full geometry from partial observations.
- Score: 20.952799332420195
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: The progressive prevalence of robots in human-suited environments has given
rise to a myriad of object manipulation techniques, in which dexterity plays a
paramount role. It is well-established that humans exhibit extraordinary
dexterity when handling objects. Such dexterity seems to derive from a robust
understanding of object properties (such as weight, size, and shape), as well
as a remarkable capacity to interact with them. Hand postures commonly
demonstrate the influence of specific regions on objects that need to be
grasped, especially when objects are partially visible. In this work, we
leverage human-like object understanding by reconstructing and completing their
full geometry from partial observations, and manipulating them using a 7-DoF
anthropomorphic robot hand. Our approach has significantly improved the
grasping success rates of baselines with only partial reconstruction by nearly
30% and achieved over 150 successful grasps with three different object
categories. This demonstrates our approach's consistent ability to predict and
execute grasping postures based on the completed object shapes from various
directions and positions in real-world scenarios. Our work opens up new
possibilities for enhancing robotic applications that require precise grasping
and manipulation skills of real-world reconstructed objects.
Related papers
- PickScan: Object discovery and reconstruction from handheld interactions [99.99566882133179]
We develop an interaction-guided and class-agnostic method to reconstruct 3D representations of scenes.
Our main contribution is a novel approach to detecting user-object interactions and extracting the masks of manipulated objects.
Compared to Co-Fusion, the only comparable interaction-based and class-agnostic baseline, this corresponds to a reduction in chamfer distance of 73%.
arXiv Detail & Related papers (2024-11-17T23:09:08Z) - Stimulating Imagination: Towards General-purpose Object Rearrangement [2.0885207827639785]
General-purpose object placement is a fundamental capability of intelligent robots.
We propose a framework named SPORT to accomplish this task.
Sport learns a diffusion-based 3D pose estimator to ensure physically-realistic results.
A set of simulation and real-world experiments demonstrate the potential of our approach to accomplish general-purpose object rearrangement.
arXiv Detail & Related papers (2024-08-03T03:53:05Z) - 3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects [13.58353565350936]
We contribute methodology to jointly estimate the geometry and pose of objects grasped by a robot.
Our method transforms the estimated geometry into the robot's coordinate frame.
We empirically evaluate our approach on a robot manipulator holding a diverse set of real-world objects.
arXiv Detail & Related papers (2024-07-14T21:02:55Z) - GraspXL: Generating Grasping Motions for Diverse Objects at Scale [30.104108863264706]
We unify the generation of hand-object grasping motions across multiple motion objectives in a policy learning framework GraspXL.
Our policy trained with 58 objects can robustly synthesize diverse grasping motions for more than 500k unseen objects with a success rate of 82.2%.
Our framework can be deployed to different dexterous hands and work with reconstructed or generated objects.
arXiv Detail & Related papers (2024-03-28T17:57:27Z) - PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large
Multimodal Models [58.33913881592706]
Humans can easily apply their intuitive physics to grasp skillfully and change grasps efficiently, even for objects they have never seen before.
This work delves into infusing such physical commonsense reasoning into robotic manipulation.
We introduce PhyGrasp, a multimodal large model that leverages inputs from two modalities: natural language and 3D point clouds.
arXiv Detail & Related papers (2024-02-26T18:57:52Z) - Full-Body Articulated Human-Object Interaction [61.01135739641217]
CHAIRS is a large-scale motion-captured f-AHOI dataset consisting of 16.2 hours of versatile interactions.
CHAIRS provides 3D meshes of both humans and articulated objects during the entire interactive process.
By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation.
arXiv Detail & Related papers (2022-12-20T19:50:54Z) - Learn to Predict How Humans Manipulate Large-sized Objects from
Interactive Motions [82.90906153293585]
We propose a graph neural network, HO-GCN, to fuse motion data and dynamic descriptors for the prediction task.
We show the proposed network that consumes dynamic descriptors can achieve state-of-the-art prediction results and help the network better generalize to unseen objects.
arXiv Detail & Related papers (2022-06-25T09:55:39Z) - Reactive Human-to-Robot Handovers of Arbitrary Objects [57.845894608577495]
We present a vision-based system that enables human-to-robot handovers of unknown objects.
Our approach combines closed-loop motion planning with real-time, temporally-consistent grasp generation.
We demonstrate the generalizability, usability, and robustness of our approach on a novel benchmark set of 26 diverse household objects.
arXiv Detail & Related papers (2020-11-17T21:52:22Z) - Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
Dexterous robotic hands are appealing for their agility and human-like morphology.
We introduce an approach for learning dexterous grasping.
Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
arXiv Detail & Related papers (2020-09-03T04:00:40Z) - Grasping Field: Learning Implicit Representations for Human Grasps [16.841780141055505]
We propose an expressive representation for human grasp modelling that is efficient and easy to integrate with deep neural networks.
We name this 3D to 2D mapping as Grasping Field, parameterize it with a deep neural network, and learn it from data.
Our generative model is able to synthesize high-quality human grasps, given only on a 3D object point cloud.
arXiv Detail & Related papers (2020-08-10T23:08:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.