UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos

• URL: http://arxiv.org/abs/2603.22264v1
• Date: Mon, 23 Mar 2026 17:49:12 GMT
• Title: UniDex: A Robot Foundation Suite for Universal Dexterous Hand Control from Egocentric Human Videos
• Abstract summary: Dexterous manipulation remains challenging due to the cost of collecting real-robot teleoperation data.<n>We present UniDex, a robot foundation suite that couples a large-scale robot-centric dataset with a unified vision-language-action (VLA) policy.<n>UniDex-Dataset, UniDex-VLA, and UniDex-Cap provide a scalable foundation suite for universal dexterous manipulation.
• Score: 65.2981273885678
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dexterous manipulation remains challenging due to the cost of collecting real-robot teleoperation data, the heterogeneity of hand embodiments, and the high dimensionality of control. We present UniDex, a robot foundation suite that couples a large-scale robot-centric dataset with a unified vision-language-action (VLA) policy and a practical human-data capture setup for universal dexterous hand control. First, we construct UniDex-Dataset, a robot-centric dataset over 50K trajectories across eight dexterous hands (6--24 DoFs), derived from egocentric human video datasets. To transform human data into robot-executable trajectories, we employ a human-in-the-loop retargeting procedure to align fingertip trajectories while preserving plausible hand-object contacts, and we operate on explicit 3D pointclouds with human hands masked to narrow kinematic and visual gaps. Second, we introduce the Function-Actuator-Aligned Space (FAAS), a unified action space that maps functionally similar actuators to shared coordinates, enabling cross-hand transfer. Leveraging FAAS as the action parameterization, we train UniDex-VLA, a 3D VLA policy pretrained on UniDex-Dataset and finetuned with task demonstrations. In addition, we build UniDex-Cap, a simple portable capture setup that records synchronized RGB-D streams and human hand poses and converts them into robot-executable trajectories to enable human-robot data co-training that reduces reliance on costly robot demonstrations. On challenging tool-use tasks across two different hands, UniDex-VLA achieves 81% average task progress and outperforms prior VLA baselines by a large margin, while exhibiting strong spatial, object, and zero-shot cross-hand generalization. Together, UniDex-Dataset, UniDex-VLA, and UniDex-Cap provide a scalable foundation suite for universal dexterous manipulation.

Related papers

• Duet: Dual-Robot Understanding via Efficient Teaching [26.80188836825586]
  DUET is a dual-robot learning framework for mobile manipulation.<n>For efficient data collection, we create a unified dual-embodiment synchronized VR-based teleoperation system.<n>We also develop a tracking pipeline that records human-human coordination and collaborative mobile manipulation priors.
  arXiv  Detail & Related papers  (2026-06-18T23:47:27Z)
• Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models [95.75234389806654]
  We present Qwen-RobotManip, a generalizable Vision-Language-Action foundation model built on Qwen-VL.<n>Qwen-RobotManip introduces a unified alignment framework across the representation, motion, and behavioral dimensions of manipulation.<n>A human-to-robot synthesis pipeline converts egocentric hand demonstrations into robot trajectories across 15 platforms.
  arXiv  Detail & Related papers  (2026-06-16T12:14:39Z)
• BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
  BifrostUMI is a portable, efficient, and robot-free data collection framework for humanoid robots.<n>We demonstrate the efficacy and versatility of the proposed framework across two distinct experimental scenarios.
  arXiv  Detail & Related papers  (2026-05-05T07:35:09Z)
• EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration [67.13034606664333]
  EgoHumanoid is the first framework to co-train a vision-language-action policy using egocentric human demonstrations.<n>A portable system for scalable human data collection is developed.
  arXiv  Detail & Related papers  (2026-02-10T18:59:03Z)
• DexImit: Learning Bimanual Dexterous Manipulation from Monocular Human Videos [56.64773686434068]
  We propose DexImit, an automated framework that converts human manipulation videos into physically plausible robot data.<n>DexImit can generate large-scale robot data based on human videos, either from the Internet or video generation models.<n>It is capable of handling diverse manipulation tasks, including tool use, long-horizon tasks, and fine-grained manipulations.
  arXiv  Detail & Related papers  (2026-02-10T18:59:02Z)
• RoboPaint: From Human Demonstration to Any Robot and Any View [9.083647729839688]
  We propose a Real-Sim-Real data collection and data editing pipeline to transform human demonstrations into robot-executable, environment-specific training data.<n>We show that retargeted dex-hand trajectories achieve an 84% success rate across 10 diverse object manipulation tasks.<n>We offer a scalable, cost-effective alternative to teleoperation with minimal performance loss for complex dexterous manipulation.
  arXiv  Detail & Related papers  (2026-02-05T05:45:12Z)
• GR-Dexter Technical Report [32.12634986073996]
  Vision-language-action (VLA) models have enabled language-conditioned, long-horizon robot manipulation.<n>We present GR-Dexter, a holistic hardware-model-data framework for VLA-based generalist manipulation on a bimanual dexterous-hand robot.
  arXiv  Detail & Related papers  (2025-12-30T13:22:16Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation [39.95361636017463]
  perioperation is a paradigm for robotic data collection that sensorizes and records human manipulation while maximizing the transferability of the data to real robots.<n>We implement this paradigm in DEXOP, a passive hand exoskeleton designed to maximize human ability to collect rich sensory (vision + tactile) data for diverse dexterous manipulation tasks in natural environments.
  arXiv  Detail & Related papers  (2025-09-04T17:57:13Z)
• RealDex: Towards Human-like Grasping for Robotic Dexterous Hand [64.33746404551343]
  We introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns.<n>RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios.
  arXiv  Detail & Related papers  (2024-02-21T14:59:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.