From Grasps to Dexterity: Large-Scale Grasp Pretraining for Dexterous Manipulation

• URL: http://arxiv.org/abs/2606.30749v1
• Date: Mon, 29 Jun 2026 18:00:22 GMT
• Title: From Grasps to Dexterity: Large-Scale Grasp Pretraining for Dexterous Manipulation
• Abstract summary: We adapt a hierarchical imitation learning framework that combines high-level hand sub-goal prediction with a low-level goal-conditioned controller.<n>We construct a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and use it to pretrain the low-level controller.<n>Across simulation and real-world experiments, our approach outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch.
• Score: 15.564266167695138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale dexterous grasp datasets encode rich priors over hand-object interaction, but their use has largely been confined to grasp generation and pick-and-place manipulation. We study whether such data can instead support functional dexterity in articulated tool use, where a robot must acquire a tool, maintain contact, and operate its functional moving parts. We adapt a hierarchical imitation learning framework that combines high-level hand sub-goal prediction with a low-level goal-conditioned controller. We construct a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and use it to pretrain the low-level controller. The controller is then fine-tuned on downstream task demonstrations. To evaluate this setting, we introduce DexCraft, a simulation benchmark with six articulated tool-use tasks requiring coordinated finger motion. Across simulation and real-world experiments, our approach outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch. In the real world, it improves full-task success by 33.3 percentage points over DP3. These results show that grasp datasets can serve not only as resources for grasp synthesis, but also as scalable pretraining data for contact-rich dexterous manipulation. Videos are shown on https://yingyuan0414.github.io/grasp2dexterity/ .

Related papers

• Pre-training Visual Dexterity in Simulation [67.17477115017896]
  We introduce a pre-training framework for dexterous manipulation that uses data entirely collected in simulation.<n>With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective.<n>We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation.
  arXiv  Detail & Related papers  (2026-08-16T20:21:33Z)
• Sparse2Act: Learning Action-Aligned Sparse 3D Representations for Cross-Domain Robot Manipulation [52.00134393320209]
  We introduce Sparse2Act, an observation-action alignment framework for pretraining sparse point-cloud encoders.<n>Masked sparse 3D tokens are trained to organize scene features around the workspace motion paired with the observation.<n>On the LIBERO-10 benchmark, our method achieves 86.9% average success after 500 fine-tuning steps.
  arXiv  Detail & Related papers  (2026-06-10T23:56:01Z)
• GCImOpt: Learning efficient goal-conditioned policies by imitating optimal trajectories [0.0]
  GCImOpt is an approach to learn efficient goal-conditioned policies by training on datasets generated by trajectory optimization.<n>We show that our trained policies can achieve high success rates and near-optimal control profiles.
  arXiv  Detail & Related papers  (2026-04-24T17:23:34Z)
• UltraDexGrasp: Learning Universal Dexterous Grasping for Bimanual Robots with Synthetic Data [70.96218513410454]
  We introduce UltraDexGrasp, a framework for universal dexterous grasping with bimanual robots.<n>The proposed data-generation pipeline integrates optimization-based grasp synthesis with planning-based demonstration generation.<n>Based on UltraDexGrasp-20M, we develop a simple yet effective grasp policy that takes point clouds as input, aggregates scene features via unidirectional attention, and predicts control commands.
  arXiv  Detail & Related papers  (2026-03-05T15:54:21Z)
• ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning [31.000965640377128]
  ABot-M0 is a framework that builds a systematic data curation pipeline.<n>It enables end-to-end transformation of heterogeneous raw data into unified, efficient representations.<n>ABot-M0 supports modular perception via a dual-stream mechanism.
  arXiv  Detail & Related papers  (2026-02-11T16:47:01Z)
• Detect Anything via Next Point Prediction [51.55967987350882]
  Rex- Omni is a 3B-scale MLLM that achieves state-of-the-art object perception performance.<n>On benchmarks like COCO and LVIS, Rex- Omni attains performance comparable to or exceeding regression-based models.
  arXiv  Detail & Related papers  (2025-10-14T17:59:54Z)
• JUICER: Data-Efficient Imitation Learning for Robotic Assembly [21.43402768760014]
  This paper proposes a pipeline for improving imitation learning performance with a small human demonstration budget. Our pipeline combines expressive policy architectures and various techniques for dataset expansion and simulation-based data augmentation. We demonstrate our pipeline on four furniture assembly tasks in simulation, enabling a manipulator to assemble up to five parts over nearly 2500 time steps.
  arXiv  Detail & Related papers  (2024-04-04T18:00:15Z)
• Dexterous Functional Grasping [39.15442658671798]
  This paper combines the best of both worlds to accomplish functional grasping for in-the-wild objects. We propose a novel application of eigengrasps to reduce the search space of RL using a small amount of human data. We find that eigengrasp action space beats baselines in simulation and outperforms hardcoded grasping in real and matches or outperforms a trained human teleoperator.
  arXiv  Detail & Related papers  (2023-12-05T18:59:23Z)
• SPOT: Scalable 3D Pre-training via Occupancy Prediction for Learning Transferable 3D Representations [76.45009891152178]
  Pretraining-finetuning approach can alleviate the labeling burden by fine-tuning a pre-trained backbone across various downstream datasets as well as tasks.<n>We show, for the first time, that general representations learning can be achieved through the task of occupancy prediction.<n>Our findings will facilitate the understanding of LiDAR points and pave the way for future advancements in LiDAR pre-training.
  arXiv  Detail & Related papers  (2023-09-19T11:13:01Z)
• DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation [10.783992625475081]
  We present a large-scale simulated dataset, DexGraspNet, for robotic dexterous grasping. We use ShadowHand, a dexterous gripper commonly seen in robotics, to generate 1.32 million grasps for 5355 objects. Compared to the previous dataset generated by GraspIt!, our dataset has not only more objects and grasps, but also higher diversity and quality.
  arXiv  Detail & Related papers  (2022-10-06T06:09:16Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.