TeleDex: Accessible Dexterous Teleoperation

• URL: http://arxiv.org/abs/2603.17065v2
• Date: Fri, 20 Mar 2026 14:00:30 GMT
• Title: TeleDex: Accessible Dexterous Teleoperation
• Abstract summary: We present TeleDex, an open-source system for intuitive teleoperation of dexterous hands and robotic manipulators using any readily available phone.<n>The system streams low-latency 6-DoF wrist poses and articulated 21-DoF hand state estimates from the phone.<n>TeleDex supports both a handheld phone-only mode and an optional 3D-printable hand-mounted interface for finger-level teleoperation.
• Score: 3.128801903176156
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite increasing dataset scale and model capacity, robot manipulation policies still struggle to generalize beyond their training distributions. As a result, deploying state-of-the-art policies in new environments, tasks, or robot embodiments often requires collecting additional demonstrations. Enabling this in real-world deployment settings requires tools that allow users to collect demonstrations quickly, affordably, and with minimal setup. We present TeleDex, an open-source system for intuitive teleoperation of dexterous hands and robotic manipulators using any readily available phone. The system streams low-latency 6-DoF wrist poses and articulated 21-DoF hand state estimates from the phone, which are retargeted to robot arms and multi-fingered hands without requiring external tracking infrastructure. TeleDex supports both a handheld phone-only mode and an optional 3D-printable hand-mounted interface for finger-level teleoperation. By lowering the hardware and setup barriers to dexterous teleoperation, TeleDex enables users to quickly collect demonstrations during deployment to support policy fine-tuning. We evaluate the system across simulation and real-world manipulation tasks, demonstrating its effectiveness as a unified scalable interface for robot teleoperation. All software and hardware designs, along with demonstration videos, are open-source and available at orayyan.com/teledex.

Related papers

• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance [52.973947477815614]
  AnyDexRT is a calibration-free method for intuitive dexterous teleoperation across human-like dexterous hands.<n>Experiments show that AnyDexRT improves quality, reduces manual tuning, and provides more intuitive and efficient control than prior methods.
  arXiv  Detail & Related papers  (2026-07-09T10:29:53Z)
• RealDexUMI: A Wearable Universal Manipulation Interface for Dexterous Robot Learning [21.875676616617838]
  We present RealDexUMI, a wearable universal manipulation interface built around a shared dexterous end-effector module.<n>A palm-side is teleoperation maps human finger inputs to robot-hand commands, enabling real-time, glove-free, intuitive, and precise hand control.
  arXiv  Detail & Related papers  (2026-06-04T11:28:23Z)
• OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer [34.13467792368733]
  We introduce OSMO, a wearable tactile glove designed for human-to-robot skill transfer.<n>We demonstrate that a robot policy trained exclusively on human demonstrations is capable of executing a challenging contact-rich manipulation task.
  arXiv  Detail & Related papers  (2025-12-09T18:56:30Z)
• Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models [50.19518681574399]
  A central challenge in real-world assistive teleoperation is for the robot to infer a wide range of human intentions from user control inputs.<n>We introduce Casper, an assistive teleoperation system that leverages commonsense knowledge embedded in pre-trained visual language models.<n>We show that Casper improves task performance, reduces human cognitive load, and achieves higher user satisfaction than direct teleoperation and assistive teleoperation baselines.
  arXiv  Detail & Related papers  (2025-06-17T17:06:43Z)
• TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation [50.261933845325636]
  We introduce TeleOpBench, a simulator-centric benchmark tailored to bimanual dexterous teleoperation.<n>Within this benchmark we implement four representative teleoperation modalities-(i) MoCap, (ii) VR device, (iii) arm-hand exoskeletons, and (iv) monocular vision tracking.
  arXiv  Detail & Related papers  (2025-05-19T06:08:53Z)
• ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation [25.679146657293778]
  Building efficient teleoperation systems across diverse robot platforms has become more crucial than ever. We develop ACE, a cross-platform visual-exoskeleton system for low-cost dexterous teleoperation. Compared to previous systems, our single system can generalize to humanoid hands, arm-hands, arm-gripper, and quadruped-gripper systems with high-precision teleoperation.
  arXiv  Detail & Related papers  (2024-08-21T17:48:31Z)
• Open-TeleVision: Teleoperation with Immersive Active Visual Feedback [17.505318269362512]
  Open-TeleVision allows operators to actively perceive the robot's surroundings in a stereoscopic manner. The system mirrors the operator's arm and hand movements on the robot, creating an immersive experience. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks.
  arXiv  Detail & Related papers  (2024-07-01T17:55:35Z)
• RealDex: Towards Human-like Grasping for Robotic Dexterous Hand [64.33746404551343]
  We introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns.<n>RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios.
  arXiv  Detail & Related papers  (2024-02-21T14:59:46Z)
• AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System [51.48191418148764]
  Vision-based teleoperation can endow robots with human-level intelligence to interact with the environment. Current vision-based teleoperation systems are designed and engineered towards a particular robot model and deploy environment. We propose AnyTeleop, a unified and general teleoperation system to support multiple different arms, hands, realities, and camera configurations within a single system.
  arXiv  Detail & Related papers  (2023-07-10T14:11:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.