Bimanual Grasp Synthesis for Dexterous Robot Hands
- URL: http://arxiv.org/abs/2411.15903v1
- Date: Sun, 24 Nov 2024 16:31:17 GMT
- Title: Bimanual Grasp Synthesis for Dexterous Robot Hands
- Abstract summary: We propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects.
The BimanGrasp algorithm generates grasp poses by optimizing an energy function.
The synthesized grasps are verified using the Isaac Gym physics simulation engine.
- Score: 1.9567015559455132
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Humans naturally perform bimanual skills to handle large and heavy objects. To enhance robots' object manipulation capabilities, generating effective bimanual grasp poses is essential. Nevertheless, bimanual grasp synthesis for dexterous hand manipulators remains underexplored. To bridge this gap, we propose the BimanGrasp algorithm for synthesizing bimanual grasps on 3D objects. The BimanGrasp algorithm generates grasp poses by optimizing an energy function that considers grasp stability and feasibility. Furthermore, the synthesized grasps are verified using the Isaac Gym physics simulation engine. These verified grasp poses form the BimanGrasp-Dataset, the first large-scale synthesized bimanual dexterous hand grasp pose dataset to our knowledge. The dataset comprises over 150k verified grasps on 900 objects, facilitating the synthesis of bimanual grasps through a data-driven approach. Last, we propose BimanGrasp-DDPM, a diffusion model trained on the BimanGrasp-Dataset. This model achieved a grasp synthesis success rate of 69.87\% and significant acceleration in computational speed compared to BimanGrasp algorithm.
Related papers
- HUGS: Guiding Unified Dexterous Grasp Synthesis Across Modes and Scales via Learned Human Priors [16.091967944555382]
We present HUGS (Human-prior-guided Unified Dexterous Grasp Synthesis), a human-prior-guided grasp synthesis across modes and scales.<n>Instead of balance directly human demonstrations, HUGS learns an object-guided human prior that captures human grasp preferences.<n>With HUGS, we synthesize 3.2M robotic grasps over 157K scenes, spanning object half-diagonal lengths from 2 cm to 30 cm and modes from two-finger to bimanual grasps.
arXiv Detail & Related papers (2026-07-06T00:01:19Z) - Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models [95.75234389806654]
We present Qwen-RobotManip, a generalizable Vision-Language-Action foundation model built on Qwen-VL.<n>Qwen-RobotManip introduces a unified alignment framework across the representation, motion, and behavioral dimensions of manipulation.<n>A human-to-robot synthesis pipeline converts egocentric hand demonstrations into robot trajectories across 15 platforms.
arXiv Detail & Related papers (2026-06-16T12:14:39Z) - BiDexGrasp: Coordinated Bimanual Dexterous Grasps across Object Geometries and Sizes [64.2030401965582]
BiDexGrasp consists of a large-scale bimanual dexterous grasp dataset and a novel generation model.<n>We construct a large-scale bimanual dexterous grasp dataset, comprising 6351 diverse objects with sizes ranging from 30 to 80 cm.<n>We introduce a bimanual-adaptive and geometry-size-adaptive dexterous grasping generation framework.
arXiv Detail & Related papers (2026-04-08T02:17:11Z) - UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data [70.96218513410454]
We introduce UltraDexGrasp, a framework for universal dexterous grasping with bimanual robots.<n>The proposed data-generation pipeline integrates optimization-based grasp synthesis with planning-based demonstration generation.<n>Based on UltraDexGrasp-20M, we develop a simple yet effective grasp policy that takes point clouds as input, aggregates scene features via unidirectional attention, and predicts control commands.
arXiv Detail & Related papers (2026-03-05T15:54:21Z) - Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos [56.510263910611684]
We tackle prehensile manipulation, in which tasks involve grasping an object before performing various post-grasp motions.<n>Human videos offer strong signals for learning the post-grasp motions, but they are less useful for learning the prerequisite grasping behaviors.<n>We present Perceive-Simulate-Imitate (PSI), a framework for training a modular manipulation policy using human video motion data.
arXiv Detail & Related papers (2026-02-13T18:59:10Z) - CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representations [53.37721117405022]
Cross-embodiment dexterous grasp synthesis refers to adaptively generating and optimizing grasps for various robotic hands.<n>We propose CEDex, a novel cross-embodiment dexterous grasp synthesis method at scale.<n>We construct the largest cross-embodiment grasp dataset to date, comprising 500K objects across four types with 20M total grasps.
arXiv Detail & Related papers (2025-09-29T12:08:04Z) - UniFucGrasp: Human-Hand-Inspired Unified Functional Grasp Annotation Strategy and Dataset for Diverse Dexterous Hands [21.591446861018238]
Dexterous grasp datasets are vital for embodied intelligence, but mostly ignore functional grasps needed for tasks like opening bottle caps or holding cup handles.<n>We establish UniFucGrasp, a universal functional grasp annotation strategy and dataset for multiple dexterous hand types.<n>Based on biomimicry, it maps natural human motions to diverse hand structures and uses geometry-based force closure to ensure functional, stable, human-like grasps.
arXiv Detail & Related papers (2025-08-05T11:37:38Z) - SSSUMO: Real-Time Semi-Supervised Submovement Decomposition [0.6499759302108926]
Submovement analysis offers valuable insights into motor control.<n>Existing methods struggle with reconstruction accuracy, computational cost, and validation.<n>We address these challenges using a semi-supervised learning framework.
arXiv Detail & Related papers (2025-07-08T21:26:25Z) - RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation [51.86515213749527]
We present RoboTwin 2.0, a scalable simulation framework that enables automated, large-scale generation of diverse and realistic data.<n>To improve sim-to-real transfer, RoboTwin 2.0 incorporates structured domain randomization along five axes: clutter, lighting, background, tabletop height and language instructions.<n>We instantiate this framework across 50 dual-arm tasks spanning five robot embodiments, and pre-collect over 100,000 domain-randomized expert trajectories.
arXiv Detail & Related papers (2025-06-22T16:26:53Z) - Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy [9.52211166462638]
Generalizable dexterous grasping with suitable grasp types is a fundamental skill for intelligent robots.
Existing automatic grasp synthesis methods are often limited to specific grasp types or object categories.
This work proposes an efficient pipeline capable of synthesizing contact-rich, penetration-free, and physically plausible grasps for any grasp type, object, and articulated hand.
arXiv Detail & Related papers (2025-04-26T07:32:59Z) - Scaling Laws of Synthetic Data for Language Models [132.67350443447611]
We introduce SynthLLM, a scalable framework that transforms pre-training corpora into diverse, high-quality synthetic datasets.
Our approach achieves this by automatically extracting and recombining high-level concepts across multiple documents using a graph algorithm.
arXiv Detail & Related papers (2025-03-25T11:07:12Z) - DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning [42.88605563822155]
We present a large-scale automated data generation system that synthesizes trajectories from human demonstrations for humanoid robots with dexterous hands.
We generate 21K demos across these tasks from just 60 source human demos.
We also present a real-to-sim-to-real pipeline and deploy it on a real-world humanoid can sorting task.
arXiv Detail & Related papers (2024-10-31T17:48:45Z) - QDGset: A Large Scale Grasping Dataset Generated with Quality-Diversity [2.095923926387536]
Quality-Diversity (QD) algorithms have been proven to make grasp sampling significantly more efficient.
We extend QDG-6DoF, a QD framework for generating object-centric grasps, to scale up the production of synthetic grasping datasets.
arXiv Detail & Related papers (2024-10-03T08:56:14Z) - Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment [56.44025052765861]
Large language models (LLMs) have revolutionized Natural Language Processing (NLP), but their size creates computational bottlenecks.
We introduce a novel approach to create accurate, sparse foundational versions of performant LLMs.
We show a total speedup on CPUs for sparse-quantized LLaMA models of up to 8.6x.
arXiv Detail & Related papers (2024-05-06T16:03:32Z) - Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large
Language Models by Extrapolating Errors from Small Models [69.76066070227452]
*Data Synthesis* is a promising way to train a small model with very little labeled data.
We propose *Synthesis Step by Step* (**S3**), a data synthesis framework that shrinks this distribution gap.
Our approach improves the performance of a small model by reducing the gap between the synthetic dataset and the real data.
arXiv Detail & Related papers (2023-10-20T17:14:25Z) - Robot Learning with Sensorimotor Pre-training [98.7755895548928]
We present a self-supervised sensorimotor pre-training approach for robotics.
Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens.
We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
arXiv Detail & Related papers (2023-06-16T17:58:10Z) - Binary stochasticity enabled highly efficient neuromorphic deep learning
achieves better-than-software accuracy [17.11946381948498]
Deep learning needs high-precision handling of forwarding signals, backpropagating errors, and updating weights.
It is challenging to implement deep learning in hardware systems that use noisy analog memristors as artificial synapses.
We propose a binary learning algorithm that modifies all elementary neural network operations.
arXiv Detail & Related papers (2023-04-25T14:38:36Z) - Automatically Prepare Training Data for YOLO Using Robotic In-Hand
Observation and Synthesis [14.034128227585143]
We propose combining robotic in-hand observation and data synthesis to enlarge the limited data set collected by the robot.
The collected and synthetic images are combined to train a deep detection neural network.
The results showed that combined observation and synthetic images led to comparable performance to manual data preparation.
arXiv Detail & Related papers (2023-01-04T04:20:08Z) - A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive
Coding Networks [65.34977803841007]
Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience.
We show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one.
arXiv Detail & Related papers (2022-11-16T00:11:04Z) - DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General
Objects Based on Simulation [10.783992625475081]
We present a large-scale simulated dataset, DexGraspNet, for robotic dexterous grasping.
We use ShadowHand, a dexterous gripper commonly seen in robotics, to generate 1.32 million grasps for 5355 objects.
Compared to the previous dataset generated by GraspIt!, our dataset has not only more objects and grasps, but also higher diversity and quality.
arXiv Detail & Related papers (2022-10-06T06:09:16Z) - BeCAPTCHA-Type: Biometric Keystroke Data Generation for Improved Bot
Detection [63.447493500066045]
This work proposes a data driven learning model for the synthesis of keystroke biometric data.
The proposed method is compared with two statistical approaches based on Universal and User-dependent models.
Our experimental framework considers a dataset with 136 million keystroke events from 168 thousand subjects.
arXiv Detail & Related papers (2022-07-27T09:26:15Z) - Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
We present a deep imitation learning framework for robotic bimanual manipulation.
A core challenge is to generalize the manipulation skills to objects in different locations.
We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
arXiv Detail & Related papers (2020-10-11T01:40:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.