A Bilateral Teleoperation Framework for Dexterous Manipulation

• URL: http://arxiv.org/abs/2606.15434v1
• Date: Sat, 13 Jun 2026 18:54:35 GMT
• Title: A Bilateral Teleoperation Framework for Dexterous Manipulation
• Abstract summary: Dexterous teleoperation requires precise arm-hand coordination, low-latency feedback, and robust interaction in real-world contact-rich environments.<n>This paper presents a modular bilateral teleoperation framework that integrates a robot-side dexterous hand and compliant robotic arm in a unified control architecture.
• Score: 3.0415970267109937
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dexterous teleoperation requires precise arm-hand coordination, low-latency feedback, and robust interaction in real-world contact-rich environments. This paper presents a modular bilateral teleoperation framework that integrates operator-side input interfaces with a robot-side dexterous hand and compliant robotic arm in a unified control architecture. The system supports position-based hand retargeting, differential arm control, multi-scale haptic feedback, and shared control for stable manipulation. We validate the framework through a real-world dexterous manipulation task, highlighting coordinated arm-hand control and contact-aware interaction. Beyond feasibility, we identify key design insights related to cross-embodiment mismatch, haptic feedback granularity, and shared control. The proposed platform provides a practical teleoperation system and a foundation for collecting high-quality demonstrations for future learning-from-demonstration research.

Related papers

• XRoboToolKit-T: Teleoperation with High Stability and Precision with Tactile Sensing for Contact-rich Manipulation [10.905283789732872]
  We propose a versatile teleoperation system with tactile-driven assistance to enable contact-rich manipulation.<n>The proposed XRoboToolKit-T teleoperation system incorporates a tactile-informed force control architecture.<n>The system achieves higher data collection efficiency and improved manipulation stability compared to state-of-the-art teleoperation without assistance.
  arXiv  Detail & Related papers  (2026-09-14T23:24:22Z)
• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• DEX-Mouse: A Low-cost Portable and Universal Interface with Force Feedback for Data Collection of Dexterous Robotic Hands [4.493178295311741]
  Data-driven dexterous hand manipulation requires large-scale, physically consistent demonstration data.<n>We present DEX-Mouse, a portable, calibration-free hand-held teleoperation interface with integrated kinesthetic feedback.
  arXiv  Detail & Related papers  (2026-04-16T13:41:07Z)
• KUKAloha: A General, Low-Cost, and Shared-Control based Teleoperation Framework for Construction Robot Arm [3.966748430445195]
  KUKAloha is a general, low-cost, and shared-control teleoperation framework for construction robot arms.<n>We implement the framework on a KUKA robot arm and conduct a usability study with representative construction manipulation tasks.
  arXiv  Detail & Related papers  (2026-03-20T16:51:30Z)
• A Vision-Based Shared-Control Teleoperation Scheme for Controlling the Robotic Arm of a Four-Legged Robot [0.9699673328328621]
  This work proposes an intuitive remote control by leveraging a vision-based pose estimation pipeline.<n>The system maps these wrist movements into robotic arm commands to control the robot's arm in real-time.<n>A trajectory planner ensures safe teleoperation by detecting and preventing collisions with both obstacles and the robotic arm itself.
  arXiv  Detail & Related papers  (2025-08-20T18:31:57Z)
• Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Models [50.19518681574399]
  A central challenge in real-world assistive teleoperation is for the robot to infer a wide range of human intentions from user control inputs.<n>We introduce Casper, an assistive teleoperation system that leverages commonsense knowledge embedded in pre-trained visual language models.<n>We show that Casper improves task performance, reduces human cognitive load, and achieves higher user satisfaction than direct teleoperation and assistive teleoperation baselines.
  arXiv  Detail & Related papers  (2025-06-17T17:06:43Z)
• Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control [72.00655365269]
  We present RoboMaster, a novel framework that models inter-object dynamics through a collaborative trajectory formulation.<n>Unlike prior methods that decompose objects, our core is to decompose the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction.<n>Our method outperforms existing approaches, establishing new state-of-the-art performance in trajectory-controlled video generation for robotic manipulation.
  arXiv  Detail & Related papers  (2025-06-02T17:57:06Z)
• TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation [50.261933845325636]
  We introduce TeleOpBench, a simulator-centric benchmark tailored to bimanual dexterous teleoperation.<n>Within this benchmark we implement four representative teleoperation modalities-(i) MoCap, (ii) VR device, (iii) arm-hand exoskeletons, and (iv) monocular vision tracking.
  arXiv  Detail & Related papers  (2025-05-19T06:08:53Z)
• Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Framework [51.39847596489193]
  Bimanual robotic manipulation is an emerging and critical topic in the robotics community.<n>We propose a novel decoupled interaction framework that considers the characteristics of different tasks in bimanual manipulation.<n>Our framework achieves outstanding performance, with a 23.5% boost over the SOTA method.
  arXiv  Detail & Related papers  (2025-03-12T09:28:41Z)
• InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint [67.6297384588837]
  We introduce a novel controllable motion generation method, InterControl, to encourage the synthesized motions maintaining the desired distance between joint pairs. We demonstrate that the distance between joint pairs for human-wise interactions can be generated using an off-the-shelf Large Language Model.
  arXiv  Detail & Related papers  (2023-11-27T14:32:33Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)
• Learning Multi-Arm Manipulation Through Collaborative Teleoperation [63.35924708783826]
  Imitation Learning (IL) is a powerful paradigm to teach robots to perform manipulation tasks. Many real-world tasks require multiple arms, such as lifting a heavy object or assembling a desk. We present Multi-Arm RoboTurk (MART), a multi-user data collection platform that allows multiple remote users to simultaneously teleoperate a set of robotic arms.
  arXiv  Detail & Related papers  (2020-12-12T05:43:43Z)
• A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU [25.451864296962288]
  We present a novel vision-based hand pose regression network (Transteleop) and an IMU-based arm tracking method. Transteleop observes the human hand through a low-cost depth camera and generates depth images of paired robot hand poses. A wearable camera holder enables simultaneous hand-arm control and facilitates the mobility of the whole teleoperation system.
  arXiv  Detail & Related papers  (2020-03-11T10:57:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.