GenDexHand: Generative Simulation for Dexterous Hands

• URL: http://arxiv.org/abs/2511.01791v1
• Date: Mon, 03 Nov 2025 17:45:38 GMT
• Title: GenDexHand: Generative Simulation for Dexterous Hands
• Abstract summary: GenDexHand is a generative simulation pipeline that autonomously produces diverse robotic tasks and environments for dexterous manipulation.<n>Our work provides a viable path toward scalable training of diverse dexterous hand behaviors in embodied intelligence.
• Score: 33.204646313894095
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data scarcity remains a fundamental bottleneck for embodied intelligence. Existing approaches use large language models (LLMs) to automate gripper-based simulation generation, but they transfer poorly to dexterous manipulation, which demands more specialized environment design. Meanwhile, dexterous manipulation tasks are inherently more difficult due to their higher degrees of freedom. Massively generating feasible and trainable dexterous hand tasks remains an open challenge. To this end, we present GenDexHand, a generative simulation pipeline that autonomously produces diverse robotic tasks and environments for dexterous manipulation. GenDexHand introduces a closed-loop refinement process that adjusts object placements and scales based on vision-language model (VLM) feedback, substantially improving the average quality of generated environments. Each task is further decomposed into sub-tasks to enable sequential reinforcement learning, reducing training time and increasing success rates. Our work provides a viable path toward scalable training of diverse dexterous hand behaviors in embodied intelligence by offering a simulation-based solution to synthetic data generation. Our website: https://winniechen2002.github.io/GenDexHand/.

Related papers

• Pre-training Visual Dexterity in Simulation [67.17477115017896]
  We introduce a pre-training framework for dexterous manipulation that uses data entirely collected in simulation.<n>With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective.<n>We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation.
  arXiv  Detail & Related papers  (2026-08-16T20:21:33Z)
• Scalable Multi-Task Data Generation via Reinforcement Learning for Language-Conditioned Bimanual Dexterous Manipulation [23.479913962930024]
  A key bottleneck in training generalist policies for dexterous manipulation is the lack of large-scale, high-quality datasets.<n>We propose a systematic RL-based data generation pipeline that integrates generalizable reward design, effective domain randomization, and language-conditioned task annotations.<n>Our experiments show that the generated data significantly improves generalization across three representative manipulation tasks.
  arXiv  Detail & Related papers  (2026-06-21T12:31:59Z)
• DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo [82.73355932907565]
  DexJoCo is a benchmark and toolkit for task-oriented dexterous manipulation.<n>We present 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning.<n>We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation.
  arXiv  Detail & Related papers  (2026-05-15T17:59:51Z)
• SoftMimicGen: A Data Generation System for Scalable Robot Learning in Deformable Object Manipulation [45.71119158785489]
  We introduce SoftMimicGen, an automated data generation pipeline for deformable object manipulation tasks.<n>We introduce a suite of high-fidelity simulation environments that encompasses a wide range of deformable objects.<n>We apply SoftMimicGen to generate datasets across the task suite, train high-performing policies from the data, and systematically analyze the data generation system.
  arXiv  Detail & Related papers  (2026-03-26T17:58:40Z)
• Imitating What Works: Simulation-Filtered Modular Policy Learning from Human Videos [56.510263910611684]
  We tackle prehensile manipulation, in which tasks involve grasping an object before performing various post-grasp motions.<n>Human videos offer strong signals for learning the post-grasp motions, but they are less useful for learning the prerequisite grasping behaviors.<n>We present Perceive-Simulate-Imitate (PSI), a framework for training a modular manipulation policy using human video motion data.
  arXiv  Detail & Related papers  (2026-02-13T18:59:10Z)
• DexImit: Learning Bimanual Dexterous Manipulation from Monocular Human Videos [56.64773686434068]
  We propose DexImit, an automated framework that converts human manipulation videos into physically plausible robot data.<n>DexImit can generate large-scale robot data based on human videos, either from the Internet or video generation models.<n>It is capable of handling diverse manipulation tasks, including tool use, long-horizon tasks, and fine-grained manipulations.
  arXiv  Detail & Related papers  (2026-02-10T18:59:02Z)
• FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generation [60.28409233931666]
  We introduce FieldGen, a field-guided data generation framework that enables scalable, diverse, and high-quality real-world data collection.<n>Experiments demonstrate that policies trained with FieldGen achieve higher success rates and improved stability compared to teleoperation-based baselines.
  arXiv  Detail & Related papers  (2025-10-23T17:47:12Z)
• GenFlowRL: Shaping Rewards with Generative Object-Centric Flow in Visual Reinforcement Learning [79.68241687396603]
  We propose GenFlowRL, which derives shaped rewards from generated flow trained from diverse cross-embodiment datasets.<n>Experiments on 10 manipulation tasks, both in simulation and real-world cross-embodiment evaluations, demonstrate that GenFlowRL effectively leverages manipulation features extracted from generated object-centric flow.
  arXiv  Detail & Related papers  (2025-08-14T20:19:20Z)
• HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning [46.57163859424286]
  This paper presents HumanoidGen, an automated task creation and demonstration collection framework.<n>Specifically, we provide spatial annotations for both assets and dexterous hands based on the atomic operations.<n>In experiments, we create a novel benchmark with augmented scenarios to evaluate the quality of the collected data.
  arXiv  Detail & Related papers  (2025-07-01T15:04:38Z)
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation [52.2244588424002]
  We present RoboTwin 2.0, a scalable framework for automated, large-scale generation of diverse and realistic data.<n>At its core is RoboTwin-OD, an object library of 731 instances across 147 categories with semantic and manipulation-relevant annotations.<n>To improve sim-to-real transfer, RoboTwin 2.0 applies structured domain randomization along five axes.
  arXiv  Detail & Related papers  (2025-06-22T16:26:53Z)
• ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning [24.675197489823898]
  We introduce ManipTrans, a novel method for transferring human bimanual skills to dexterous robotic hands in simulation.<n>Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency.<n>We also create DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing.
  arXiv  Detail & Related papers  (2025-03-27T17:50:30Z)
• DexterityGen: Foundation Controller for Unprecedented Dexterity [67.15251368211361]
  Teaching robots dexterous manipulation skills, such as tool use, presents a significant challenge.<n>Current approaches can be broadly categorized into two strategies: human teleoperation (for imitation learning) and sim-to-real reinforcement learning.<n>We introduce DexterityGen, which uses RL to pretrain large-scale dexterous motion primitives, such as in-hand rotation or translation.<n>In the real world, we use human teleoperation as a prompt to the controller to produce highly dexterous behavior.
  arXiv  Detail & Related papers  (2025-02-06T18:49:35Z)
• DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning [42.88605563822155]
  We present a large-scale automated data generation system that synthesizes trajectories from human demonstrations for humanoid robots with dexterous hands.<n>We generate 21K demos across these tasks from just 60 source human demos.<n>We also present a real-to-sim-to-real pipeline and deploy it on a real-world humanoid can sorting task.
  arXiv  Detail & Related papers  (2024-10-31T17:48:45Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.