DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization

• URL: http://arxiv.org/abs/2603.05861v1
• Date: Fri, 06 Mar 2026 03:36:33 GMT
• Title: DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization
• Abstract summary: High-fidelity teleoperation of dexterous robotic hands is essential for bringing robots into unstructured domestic environments.<n>We develop a lightweight and cost-effective teleoperation system leveraging surface electromyography (sEMG)<n>We show that our system achieves high precision in diverse teleoperation tasks.
• Score: 5.972698171130012
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-fidelity teleoperation of dexterous robotic hands is essential for bringing robots into unstructured domestic environments. However, existing teleoperation systems often face a trade-off between performance and portability: vision-based capture systems are constrained by costs and line-of-sight requirements, while mechanical exoskeletons are bulky and physically restrictive. In this paper, we present DexEMG, a lightweight and cost-effective teleoperation system leveraging surface electromyography (sEMG) to bridge the gap between human intent and robotic execution. We first collect a synchronized dataset of sEMG signals and hand poses via a MoCap glove to train EMG2Pose, a neural network capable of continuously predicting hand kinematics directly from muscle activity. To ensure seamless control, we develop a robust hand retargeting algorithm that maps the predicted poses onto a multi-fingered dexterous hand in real-time. Experimental results demonstrate that DexEMG achieves high precision in diverse teleoperation tasks. Notably, our system exhibits strong generalization capabilities across novel objects and complex environments without the need for intensive individual-specific recalibration. This work offers a scalable and intuitive interface for both general-purpose robotic manipulation and assistive technologies.

Related papers

• Real-Time sEMG-Based Telecontrol of an Assistive Robotic Arm Using a 1D Convolutional Neural Network [0.30786914102688595]
  This work proposes a real-time sEMG-based interface for the teleoperation of an assistive robotic arm.<n>The system relies on four-channel sEMG acquisition, signal preprocessing, segmentation into sliding windows, and classification using a one-dimensional convolutional neural network (CNN)<n>The system exhibits stable real-time behavior, with an average latency of approximately 0.32 s consistent with the chosen windowing strategy.
  arXiv  Detail & Related papers  (2026-07-14T15:34:28Z)
• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• ActiveGlasses: Learning Manipulation with Active Vision from Ego-centric Human Demonstration [51.69384671495837]
  ActiveGlasses is a system for learning robot manipulation from ego-centric human demonstrations with active vision.<n>A stereo camera mounted on smart glasses serves as the sole perception device for both data collection and policy inference.<n>To enable zero-transfer, we extract object trajectories from demonstrations and use an object-centric point-cloud policy to jointly predict manipulation and head movement.
  arXiv  Detail & Related papers  (2026-04-09T17:59:08Z)
• HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation [74.34984994596813]
  HEX is a state-centric framework for coordinated manipulation on humanoid robots.<n>It incorporates a humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments.<n>It achieves state-of-the-art performance in task success rate and generalization.
  arXiv  Detail & Related papers  (2026-04-09T09:01:43Z)
• Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery [64.82869118243723]
  We present a framework that integrates EEG-based visual and motor imagery (VI/MI) with robotic control to enable real-time, intention-driven grasping and placement.<n>Motivated by the promise of BCI-driven robotics, this system bridges neural signals with physical control by deploying offline-pretrained decoders in a zero-shot manner.
  arXiv  Detail & Related papers  (2026-03-03T17:41:42Z)
• Interpretable Multimodal Gesture Recognition for Drone and Mobile Robot Teleoperation via Log-Likelihood Ratio Fusion [14.332919759770645]
  Vision-based gesture recognition has been explored as one method for hands-free teleoperation.<n>We propose a multimodal gesture recognition framework that integrates inertial data from Apple Watches on both wrists with capacitive sensing signals from custom gloves.<n>We show that our framework achieves performance comparable to a state-of-the-art vision-based baseline.
  arXiv  Detail & Related papers  (2026-02-27T05:52:04Z)
• CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation [70.5382178207975]
  hIsight Perturbation (CHIP) is a plug-and-play module that enables controllable end-effector stiffness.<n>CHIP is easy to implement and requires neither data augmentation nor additional reward tuning.<n>We show that a generalist motion-tracking controller trained with CHIP can perform a diverse set of forceful manipulation tasks.
  arXiv  Detail & Related papers  (2025-12-16T18:56:04Z)
• TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation [50.261933845325636]
  We introduce TeleOpBench, a simulator-centric benchmark tailored to bimanual dexterous teleoperation.<n>Within this benchmark we implement four representative teleoperation modalities-(i) MoCap, (ii) VR device, (iii) arm-hand exoskeletons, and (iv) monocular vision tracking.
  arXiv  Detail & Related papers  (2025-05-19T06:08:53Z)
• Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation [34.47272224723296]
  We present Taccel, a high-performance simulation platform that integrates IPC and ABD to model robots, tactile sensors, and objects with both accuracy and unprecedented speed.<n>Unlike previous simulators that operate at sub-real-time speeds with limited parallelization, Taccel provides precise physics simulation and realistic tactile signals.<n>These capabilities position Taccel as a powerful tool for scaling up tactile robotics research and development, potentially transforming how robots interact with and understand their physical environment.
  arXiv  Detail & Related papers  (2025-04-17T12:57:11Z)
• HACTS: a Human-As-Copilot Teleoperation System for Robot Learning [47.9126187195398]
  We introduce HACTS (Human-As-Copilot Teleoperation System), a novel system that establishes bilateral, real-time joint synchronization between a robot arm and teleoperation hardware.<n>This simple yet effective feedback mechanism, akin to a steering wheel in autonomous vehicles, enables the human copilot to intervene seamlessly while collecting action-correction data for future learning.
  arXiv  Detail & Related papers  (2025-03-31T13:28:13Z)
• emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation [12.566524562446467]
  Reliable and always-available hand pose inference could yield new and intuitive control schemes for human-computer interactions.<n>Wearable wrist-based surface electromyography (sEMG) presents a promising alternative.<n>emg2pose is the largest publicly available dataset of high-quality hand pose labels and wrist sEMG recordings.
  arXiv  Detail & Related papers  (2024-12-02T23:39:37Z)
• ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation [25.679146657293778]
  Building efficient teleoperation systems across diverse robot platforms has become more crucial than ever. We develop ACE, a cross-platform visual-exoskeleton system for low-cost dexterous teleoperation. Compared to previous systems, our single system can generalize to humanoid hands, arm-hands, arm-gripper, and quadruped-gripper systems with high-precision teleoperation.
  arXiv  Detail & Related papers  (2024-08-21T17:48:31Z)
• AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System [51.48191418148764]
  Vision-based teleoperation can endow robots with human-level intelligence to interact with the environment. Current vision-based teleoperation systems are designed and engineered towards a particular robot model and deploy environment. We propose AnyTeleop, a unified and general teleoperation system to support multiple different arms, hands, realities, and camera configurations within a single system.
  arXiv  Detail & Related papers  (2023-07-10T14:11:07Z)
• A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU [25.451864296962288]
  We present a novel vision-based hand pose regression network (Transteleop) and an IMU-based arm tracking method. Transteleop observes the human hand through a low-cost depth camera and generates depth images of paired robot hand poses. A wearable camera holder enables simultaneous hand-arm control and facilitates the mobility of the whole teleoperation system.
  arXiv  Detail & Related papers  (2020-03-11T10:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.