Bimanual Grasp Synthesis for Dexterous Robot Hands

• URL: http://arxiv.org/abs/2411.15903v1
• Date: Sun, 24 Nov 2024 16:31:17 GMT
• Title: Bimanual Grasp Synthesis for Dexterous Robot Hands
• Authors: Yanming Shao, Chenxi Xiao,
• Abstract summary: We propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function. The synthesized grasps are verified using the Isaac Gym physics simulation engine.
• Score: 1.9567015559455132
• License:
• Abstract: Humans naturally perform bimanual skills to handle large and heavy objects. To enhance robots' object manipulation capabilities, generating effective bimanual grasp poses is essential. Nevertheless, bimanual grasp synthesis for dexterous hand manipulators remains underexplored. To bridge this gap, we propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function that considers grasp stability and feasibility. Furthermore, the synthesized grasps are verified using the Isaac Gym physics simulation engine. These verified grasp poses form the BimanGrasp-Dataset, the first large-scale synthesized bimanual dexterous hand grasp pose dataset to our knowledge. The dataset comprises over 150k verified grasps on 900 objects, facilitating the synthesis of bimanual grasps through a data-driven approach. Last, we propose BimanGrasp-DDPM, a diffusion model trained on the BimanGrasp-Dataset. This model achieved a grasp synthesis success rate of 69.87\% and significant acceleration in computational speed compared to BimanGrasp algorithm.

Related papers

• QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversity [2.095923926387536]
  Quality-Diversity (QD) algorithms have been proven to make grasp sampling significantly more efficient. We extend QDG-6DoF, a QD framework for generating object-centric grasps, to scale up the production of synthetic grasping datasets.
  arXiv  Detail & Related papers  (2024-10-03T08:56:14Z)
• Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment [56.44025052765861]
  Large language models (LLMs) have revolutionized Natural Language Processing (NLP), but their size creates computational bottlenecks. We introduce a novel approach to create accurate, sparse foundational versions of performant LLMs. We show a total speedup on CPUs for sparse-quantized LLaMA models of up to 8.6x.
  arXiv  Detail & Related papers  (2024-05-06T16:03:32Z)
• Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models [69.76066070227452]
  *Data Synthesis* is a promising way to train a small model with very little labeled data. We propose *Synthesis Step by Step* (**S3**), a data synthesis framework that shrinks this distribution gap. Our approach improves the performance of a small model by reducing the gap between the synthetic dataset and the real data.
  arXiv  Detail & Related papers  (2023-10-20T17:14:25Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy [17.11946381948498]
  Deep learning needs high-precision handling of forwarding signals, backpropagating errors, and updating weights. It is challenging to implement deep learning in hardware systems that use noisy analog memristors as artificial synapses. We propose a binary learning algorithm that modifies all elementary neural network operations.
  arXiv  Detail & Related papers  (2023-04-25T14:38:36Z)
• Automatically Prepare Training Data for YOLO Using Robotic In-Hand Observation and Synthesis [14.034128227585143]
  We propose combining robotic in-hand observation and data synthesis to enlarge the limited data set collected by the robot. The collected and synthetic images are combined to train a deep detection neural network. The results showed that combined observation and synthetic images led to comparable performance to manual data preparation.
  arXiv  Detail & Related papers  (2023-01-04T04:20:08Z)
• A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks [65.34977803841007]
  Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. We show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one.
  arXiv  Detail & Related papers  (2022-11-16T00:11:04Z)
• DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation [10.783992625475081]
  We present a large-scale simulated dataset, DexGraspNet, for robotic dexterous grasping. We use ShadowHand, a dexterous gripper commonly seen in robotics, to generate 1.32 million grasps for 5355 objects. Compared to the previous dataset generated by GraspIt!, our dataset has not only more objects and grasps, but also higher diversity and quality.
  arXiv  Detail & Related papers  (2022-10-06T06:09:16Z)
• BeCAPTCHA-Type: Biometric Keystroke Data Generation for Improved Bot Detection [63.447493500066045]
  This work proposes a data driven learning model for the synthesis of keystroke biometric data. The proposed method is compared with two statistical approaches based on Universal and User-dependent models. Our experimental framework considers a dataset with 136 million keystroke events from 168 thousand subjects.
  arXiv  Detail & Related papers  (2022-07-27T09:26:15Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.