RealDexUMI: A Wearable Universal Manipulation Interface for Dexterous Robot Learning

• URL: http://arxiv.org/abs/2606.06033v2
• Date: Sat, 06 Jun 2026 19:36:14 GMT
• Title: RealDexUMI: A Wearable Universal Manipulation Interface for Dexterous Robot Learning
• Abstract summary: We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module.<n>A palm-side is teleoperation maps human finger inputs to robot-hand commands, enabling real-time, glove-free, intuitive, and precise hand control.
• Score: 21.875676616617838
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning dexterous manipulation requires demonstrations that preserve fine hand-object interactions while remaining executable at deployment. Existing pipelines either lose deployable dexterity through retargeting or embodiment conversion, or rely on robot-specific teleoperation that is costly to scale and often lacks intuitive, contact-aware control for dexterous data collection. We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module that integrates a lightweight dexterous hand, in-hand vision, and fingertip tactile sensing. A palm-side isomorphic teleoperation glove maps human finger inputs to robot-hand joint commands, enabling real-time, retargeting-free, intuitive, and precise hand control. The shared hand and sensing modules yield zero-gap end-effector data, with matched in-hand observations, tactile signals, contacts, and hand actions between collection and deployment. Across eight real-robot tasks spanning fine-grained, contact-rich, long-horizon, and bimanual manipulation, policies trained on RealDexUMI data achieve an average success rate of 88.75%, generalize to unseen initial poses, and transfer across three embodiments. Website: https://research.beingbeyond.com/realdexumi

Related papers

• AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning [25.241504281004243]
  We present AdvDex, a framework for learning dexterous manipulation from human and robot demonstrations.<n>First, we introduce OmniShare, a large-scale multimodal dataset of human manipulation demonstrations.<n>Second, we propose the Joint-Aligned Action Space (JAAS), a canonical action representation comprising an $mathrmSE(3)$ wrist pose and 15 finger joints.<n>Third, we use domain-adversarial learning to reduce embodiment-specific information in the learned visual representation.
  arXiv  Detail & Related papers  (2026-08-14T07:19:06Z)
• NestDex: Nested Policy Learning with Copilot Assisted Teleoperation for Dexterous Manipulation [3.901334865327756]
  Dexterous manipulation promises substantially richer robot interaction with the physical world.<n>But learning these behaviours remains constrained by the difficulty of collecting consistent, complete-task demonstrations.<n>We introduce NestDex, a nested policy-learning framework that reduces this burden by using learned hand skills to assist demonstration collection.
  arXiv  Detail & Related papers  (2026-08-13T15:25:39Z)
• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance [52.973947477815614]
  AnyDexRT is a calibration-free method for intuitive dexterous teleoperation across human-like dexterous hands.<n>Experiments show that AnyDexRT improves quality, reduces manual tuning, and provides more intuitive and efficient control than prior methods.
  arXiv  Detail & Related papers  (2026-07-09T10:29:53Z)
• UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos [65.2981273885678]
  Dexterous manipulation remains challenging due to the cost of collecting real-robot teleoperation data.<n>We present UniDex, a robot foundation suite that couples a large-scale robot-centric dataset with a unified vision-language-action (VLA) policy.<n>UniDex-Dataset, UniDex-VLA, and UniDex-Cap provide a scalable foundation suite for universal dexterous manipulation.
  arXiv  Detail & Related papers  (2026-03-23T17:49:12Z)
• DexImit: Learning Bimanual Dexterous Manipulation from Monocular Human Videos [56.64773686434068]
  We propose DexImit, an automated framework that converts human manipulation videos into physically plausible robot data.<n>DexImit can generate large-scale robot data based on human videos, either from the Internet or video generation models.<n>It is capable of handling diverse manipulation tasks, including tool use, long-horizon tasks, and fine-grained manipulations.
  arXiv  Detail & Related papers  (2026-02-10T18:59:02Z)
• GR-Dexter Technical Report [32.12634986073996]
  Vision-language-action (VLA) models have enabled language-conditioned, long-horizon robot manipulation.<n>We present GR-Dexter, a holistic hardware-model-data framework for VLA-based generalist manipulation on a bimanual dexterous-hand robot.
  arXiv  Detail & Related papers  (2025-12-30T13:22:16Z)
• MILE: A Mechanically Isomorphic Exoskeleton Data Collection System with Fingertip Visuotactile Sensing for Dexterous Manipulation [17.138615434309575]
  Existing data-collection pipelines suffer from inaccurate motion manipulation, low data-collection efficiency, and missing high-resolution tactile sensing.<n>We address this gap with MILE, a mechanically tele-operation and data-collection system co-designed from human hand to robotic hand.
  arXiv  Detail & Related papers  (2025-11-29T05:34:39Z)
• Grasp Like Humans: Learning Generalizable Multi-Fingered Grasping from Human Proprioceptive Sensorimotor Integration [26.351720551267846]
  Tactile and kinesthetic perceptions are crucial for human dexterous manipulation, enabling reliable grasping of objects via sensorimotor integration.<n>We propose a novel glove-mediated tactile-kinematic perception-prediction framework for grasp skill transfer from human intuitive and natural operation to robotic execution based on imitation learning.
  arXiv  Detail & Related papers  (2025-09-10T07:44:12Z)
• DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation [78.60543357822957]
  Dexterous manipulation with contact-rich interactions is crucial for advanced robotics.<n>We introduce DexHandDiff, an interaction-aware diffusion planning framework for adaptive dexterous manipulation.<n>Our framework achieves an average of 70.7% success rate on goal adaptive dexterous tasks, highlighting its robustness and flexibility in contact-rich manipulation.
  arXiv  Detail & Related papers  (2024-11-27T18:03:26Z)
• RealDex: Towards Human-like Grasping for Robotic Dexterous Hand [64.33746404551343]
  We introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns.<n>RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios.
  arXiv  Detail & Related papers  (2024-02-21T14:59:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.