DMFC-GraspNet: Differentiable Multi-Fingered Robotic Grasp Generation in
Cluttered Scenes
- URL: http://arxiv.org/abs/2308.00456v2
- Date: Wed, 16 Aug 2023 20:53:23 GMT
- Title: DMFC-GraspNet: Differentiable Multi-Fingered Robotic Grasp Generation in
Cluttered Scenes
- Authors: Philipp Bl\"attner, Johannes Brand, Gerhard Neumann, Ngo Anh Vien
- Abstract summary: Multi-fingered robotic grasping can potentially perform complex object manipulation.
Current techniques for multi-fingered robotic grasping frequently predict only a single grasp for each inference time.
This paper proposes a differentiable multi-fingered grasp generation network (DMFC-GraspNet) with three main contributions to address this challenge.
- Score: 22.835683657191936
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Robotic grasping is a fundamental skill required for object manipulation in
robotics. Multi-fingered robotic hands, which mimic the structure of the human
hand, can potentially perform complex object manipulation. Nevertheless,
current techniques for multi-fingered robotic grasping frequently predict only
a single grasp for each inference time, limiting computational efficiency and
their versatility, i.e. unimodal grasp distribution. This paper proposes a
differentiable multi-fingered grasp generation network (DMFC-GraspNet) with
three main contributions to address this challenge. Firstly, a novel neural
grasp planner is proposed, which predicts a new grasp representation to enable
versatile and dense grasp predictions. Secondly, a scene creation and label
mapping method is developed for dense labeling of multi-fingered robotic hands,
which allows a dense association of ground truth grasps. Thirdly, we propose to
train DMFC-GraspNet end-to-end using using a forward-backward automatic
differentiation approach with both a supervised loss and a differentiable
collision loss and a generalized Q 1 grasp metric loss. The proposed approach
is evaluated using the Shadow Dexterous Hand on Mujoco simulation and ablated
by different choices of loss functions. The results demonstrate the
effectiveness of the proposed approach in predicting versatile and dense
grasps, and in advancing the field of multi-fingered robotic grasping.
Related papers
- Multi-Transmotion: Pre-trained Model for Human Motion Prediction [68.87010221355223]
Multi-Transmotion is an innovative transformer-based model designed for cross-modality pre-training.
Our methodology demonstrates competitive performance across various datasets on several downstream tasks.
arXiv Detail & Related papers (2024-11-04T23:15:21Z) - Decomposed Vector-Quantized Variational Autoencoder for Human Grasp Generation [27.206656215734295]
We propose a novel Decomposed Vector-Quantized Variational Autoencoder (DVQ-VAE) to generate realistic human grasps.
Part-aware decomposed architecture facilitates more precise management of the interaction between each component of hand and object.
Our model achieved about 14.1% relative improvement in the quality index compared to the state-of-the-art methods in four widely-adopted benchmarks.
arXiv Detail & Related papers (2024-07-19T06:41:16Z) - CMG-Net: An End-to-End Contact-Based Multi-Finger Dexterous Grasping
Network [25.879649629474212]
We present an effective end-to-end network, CMG-Net, for grasping unknown objects in a cluttered environment.
We create a synthetic grasp dataset that consists of five thousand cluttered scenes, 80 object categories, and 20 million annotations.
Our work significantly outperforms the state-of-the-art for three-finger robotic hands.
arXiv Detail & Related papers (2023-03-23T11:29:31Z) - Deep Functional Predictive Control for Strawberry Cluster Manipulation
using Tactile Prediction [6.365634303789478]
This paper introduces a novel approach to address the problem of Physical Robot Interaction (PRI) during robot pushing tasks.
The approach uses a data-driven forward model based on tactile predictions to inform the controller about potential future movements of the object being pushed.
arXiv Detail & Related papers (2023-03-09T16:31:35Z) - Hybrid Physical Metric For 6-DoF Grasp Pose Detection [46.84694505427047]
We propose a hybrid physical metric to generate elaborate confidence scores for 6-DoF grasp pose detection.
To learn the new confidence scores effectively, we design a multi-resolution network called Flatness Gravity Collision GraspNet.
Our method achieves 90.5% success rate in real-world cluttered scenes.
arXiv Detail & Related papers (2022-06-22T14:35:48Z) - Intelligent Trajectory Design for RIS-NOMA aided Multi-robot
Communications [59.34642007625687]
The goal is to maximize the sum-rate of whole trajectories for multi-robot system by jointly optimizing trajectories and NOMA decoding orders of robots.
An integrated machine learning (ML) scheme is proposed, which combines long short-term memory (LSTM)-autoregressive integrated moving average (ARIMA) model and dueling double deep Q-network (D$3$QN) algorithm.
arXiv Detail & Related papers (2022-05-03T17:14:47Z) - Model Predictive Control for Fluid Human-to-Robot Handovers [50.72520769938633]
Planning motions that take human comfort into account is not a part of the human-robot handover process.
We propose to generate smooth motions via an efficient model-predictive control framework.
We conduct human-to-robot handover experiments on a diverse set of objects with several users.
arXiv Detail & Related papers (2022-03-31T23:08:20Z) - Enhancing Multi-Robot Perception via Learned Data Association [37.866254392010454]
We address the multi-robot collaborative perception problem, specifically in the context of multi-view infilling for distributed semantic segmentation.
We propose the Multi-Agent Infilling Network: an neural architecture that can be deployed to each agent in a robotic swarm.
Specifically, each robot is in charge of locally encoding and decoding visual information, and an neural mechanism allows for an uncertainty-aware and context-based exchange of intermediate features.
arXiv Detail & Related papers (2021-07-01T22:45:26Z) - Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for
Multi-Robot Systems [92.26462290867963]
Kimera-Multi is the first multi-robot system that is robust and capable of identifying and rejecting incorrect inter and intra-robot loop closures.
We demonstrate Kimera-Multi in photo-realistic simulations, SLAM benchmarking datasets, and challenging outdoor datasets collected using ground robots.
arXiv Detail & Related papers (2021-06-28T03:56:40Z) - Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic
Simultaneous Localization and Mapping [57.173793973480656]
We present the first fully distributed multi-robot system for dense metric-semantic SLAM.
Our system, dubbed Kimera-Multi, is implemented by a team of robots equipped with visual-inertial sensors.
Kimera-Multi builds a 3D mesh model of the environment in real-time, where each face of the mesh is annotated with a semantic label.
arXiv Detail & Related papers (2020-11-08T21:38:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.