DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations

• URL: http://arxiv.org/abs/2602.21811v1
• Date: Wed, 25 Feb 2026 11:38:07 GMT
• Title: DexRepNet++: Learning Dexterous Robotic Manipulation with Geometric and Spatial Hand-Object Representations
• Abstract summary: We propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects.<n>Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and experiments are conducted.<n>For grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories.<n>For the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%
• Score: 30.95930862562584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic dexterous manipulation is a challenging problem due to high degrees of freedom (DoFs) and complex contacts of multi-fingered robotic hands. Many existing deep reinforcement learning (DRL) based methods aim at improving sample efficiency in high-dimensional output action spaces. However, existing works often overlook the role of representations in achieving generalization of a manipulation policy in the complex input space during the hand-object interaction. In this paper, we propose DexRep, a novel hand-object interaction representation to capture object surface features and spatial relations between hands and objects for dexterous manipulation skill learning. Based on DexRep, policies are learned for three dexterous manipulation tasks, i.e. grasping, in-hand reorientation, bimanual handover, and extensive experiments are conducted to verify the effectiveness. In simulation, for grasping, the policy learned with 40 objects achieves a success rate of 87.9% on more than 5000 unseen objects of diverse categories, significantly surpassing existing work trained with thousands of objects; for the in-hand reorientation and handover tasks, the policies also boost the success rates and other metrics of existing hand-object representations by 20% to 40%. The grasp policies with DexRep are deployed to the real world under multi-camera and single-camera setups and demonstrate a small sim-to-real gap.

Related papers

• One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry [49.066054862620014]
  Learning from human demonstrations has emerged as a scalable alternative to robot teleoperation.<n>Recent sim-to-real RL methods incorporate such priors, but often omit rewards that explicitly incentivize precise contact.<n>We propose DemoMimic, a policy that manipulates objects by focusing on their geometry local to the contact points.
  arXiv  Detail & Related papers  (2026-09-01T23:04:35Z)
• Concurrent Prehensile and Nonprehensile Manipulation: A Practical Approach to Multi-Stage Dexterous Tasks [23.52755440099898]
  We present DexMulti, a sample-efficient approach for real-world dexterous multi-task manipulation.<n>Rather than learning monolithic policies, our method retrieves demonstrated skills based on current object geometry.<n>Our approach achieves an average success rate of 66% on training objects with only 3-4 demonstrations per object, outperforming diffusion policy baselines by 2-3x.
  arXiv  Detail & Related papers  (2026-03-12T08:25:21Z)
• Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstruction [24.49384094440561]
  We propose VIDEOMANIP, a device-free framework that learns dexterous manipulation directly from RGB human videos.<n>In simulation, the learned grasping model achieves a 70.25% success rate across 20 diverse objects using the Inspire Hand.<n>In the real world, manipulation policies trained from RGB videos achieve an average 62.86% success rate across seven tasks using the LEAP Hand.
  arXiv  Detail & Related papers  (2026-02-09T18:56:02Z)
• Learning Dexterous Object Handover [4.351636062759616]
  In this work, we demonstrate the use of Reinforcement Learning (RL) for dexterous object handover between two multi-finger hands.<n>Key to this task is the use of a novel reward function based on dual quaternions to minimize the rotation distance.<n>The results demonstrate that the trained policy successfully perform this task, achieving a total success rate of 94% in the best-case scenario after 100 experiments.
  arXiv  Detail & Related papers  (2025-06-20T08:22:46Z)
• Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids [56.892520712892804]
  We introduce a practical sim-to-real RL recipe that trains a humanoid robot to perform three dexterous manipulation tasks.<n>We demonstrate high success rates on unseen objects and robust, adaptive policy behaviors.
  arXiv  Detail & Related papers  (2025-02-27T18:59:52Z)
• Twisting Lids Off with Two Hands [82.21668778600414]
  We show how policies trained in simulation can be effectively and efficiently transferred to the real world. Specifically, we consider the problem of twisting lids of various bottle-like objects with two hands. This is the first sim-to-real RL system that enables such capabilities on bimanual multi-fingered hands.
  arXiv  Detail & Related papers  (2024-03-04T18:59:30Z)
• DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrations [51.87067543670535]
  We propose a robot-learning system that can take a small number of human demonstrations and learn to grasp unseen object poses. We train a dexterous grasping policy that takes the point clouds of the object as input and predicts continuous actions to grasp objects from different initial robot states. The policy learned from our dataset can generalize well on unseen object poses in both simulation and the real world.
  arXiv  Detail & Related papers  (2022-09-28T17:51:49Z)
• Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning [108.08083976908195]
  We show that policies learned by existing reinforcement learning algorithms can in fact be generalist. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms single-object specialist policies.
  arXiv  Detail & Related papers  (2021-11-04T17:59:56Z)
• A Framework for Efficient Robotic Manipulation [79.10407063260473]
  We show that a single robotic arm can learn sparse-reward manipulation policies from pixels. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels.
  arXiv  Detail & Related papers  (2020-12-14T22:18:39Z)
• Learning Dexterous Grasping with Object-Centric Visual Affordances [86.49357517864937]
  Dexterous robotic hands are appealing for their agility and human-like morphology. We introduce an approach for learning dexterous grasping. Our key idea is to embed an object-centric visual affordance model within a deep reinforcement learning loop.
  arXiv  Detail & Related papers  (2020-09-03T04:00:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.