DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation

• URL: http://arxiv.org/abs/2605.12182v1
• Date: Tue, 12 May 2026 14:25:00 GMT
• Title: DexTwist: Dexterous Hand Retargeting for Twist Motion via Mixed Reality-based Teleoperation
• Abstract summary: Dexterous teleoperation via Mixed Reality (MR)-based interfaces offer a scalable paradigm for transferring human manipulation skills to robot hands.<n>We propose DexTwist, a functional twist-retargeting framework for MR-based dexterous teleoperation.<n>We show that DexTwist improves turning angle tracking and screw axis stability compared with a vector-based baseline.
• Score: 7.817721592278407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dexterous teleoperation via Mixed Reality (MR)-based interfaces offers a scalable paradigm for transferring human manipulation skills to dexterous robot hands. However, conventional retargeting approaches that minimize kinematic dissimilarity (e.g., joint angle or fingertip position error) often fail in contact-rich rotational manipulation, such as cap opening, key turning, and bolt screwing. This failure stems from the embodiment gap: mismatched link lengths, joint axes/limits, and fingertip geometry can cause direct pose imitation to induce tangential fingertip sliding rather than stable object rotation, resulting in screw axis drift, contact slip, and grasp instability. To address this, we propose DexTwist, a functional twist-retargeting framework for MR-based dexterous teleoperation. DexTwist detects a tripod pinch, estimates the operator's intended screw axis and twist magnitude, and applies a real-time residual joint-space refinement that tracks turning progress while regularizing the robot tripod geometry. The refinement minimizes a virtual-object objective defined by turning angle, screw axis consistency, fingertip closure, and tripod stability. Simulation and real-world experiments show that DexTwist improves turning angle tracking and screw axis stability compared with a vector-based retargeting baseline.

Related papers

• C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video [47.458936125974226]
  C2Dex is a video-to-dexterous-manipulation framework built around a shared interaction.<n> stable object-side contacts recovered by noisy frame-wise observations in the canonical object space.<n>C2Dex achieves end-to-end trajectory success rates of 57.78% and 26.67%.
  arXiv  Detail & Related papers  (2026-08-07T09:54:00Z)
• Towards Human-level Dexterous Teleoperation [62.77371751776912]
  Human-level dexterous teleoperation requires precise control of object motion via dynamic hand-object contact.<n>We introduce TeleDexter, a hand-object co-tracking controller that maps operator intent into learned, low-level contact execution.<n>We evaluate TeleDexter on seven challenging dexterous teleoperation tasks spanning object reorientation and long-horizon tool use across two dexterous hands.
  arXiv  Detail & Related papers  (2026-07-13T12:36:47Z)
• Seeing Through Occlusion: Deterministic Arm Kinematic Correction for Robot Teleoperation [6.979012378888037]
  This paper presents an Arm Kinematic Correction (AKC) method that improves depth estimation by enforcing geometric constraints based on constant arm lengths.<n> Experimental validation against a Vicon reference system demonstrates reliable performance for both static and dynamic joint motions.<n>The results show that AKC enhances robustness and preserves anatomical consistency under long-duration, severe self-occlusion, even when paired with less reliable temporal filters.
  arXiv  Detail & Related papers  (2026-06-17T16:20:10Z)
• TacSE3: Equivariant SE(3) Motion Estimation from Low-Texture Visuotactile Images for In-Gripper Tracking and Compensation [21.692486374633418]
  TacSE3 is a tactile motion-estimation pipeline that converts low-texture visuotactile observations into a decoupled three-dimensional force field.<n>The method derives planar translation from contact-centroid motion and estimates rotation primarily from shear-related tactile responses.<n>Experiments with paired DM-Tac fingertip sensors show that dual-sensor sensing reduces translation-rotation ambiguity, supports rotation tracking across axes and object geometries, and provides a lightweight compensation signal.
  arXiv  Detail & Related papers  (2026-05-18T06:38:06Z)
• MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons [56.68975315643491]
  We present the first fully end-to-end framework in which Videoto-Pose and Pose-to-Rotation are learnable and optimized jointly.<n>We show that our method reduces rotation error from 17 degrees to 10 degrees, and to 6.54 degrees on unseen skeletons, while achieving 20x faster inference than mesh-based pipelines.
  arXiv  Detail & Related papers  (2026-04-30T17:16:38Z)
• Periodic Steady-State Control of a Handkerchief-Spinning Task Using a Parallel Anti-Parallelogram Tendon-driven Wrist [37.0667004392926]
  We develop an intuitive dexterous wrist based on a parallel anti-parallelogram tendon-driven structure.<n>We then develop a particle-spring model of the handkerchief for control-oriented abstraction and strategy evaluation.<n>Results demonstrate that integrating control-oriented modeling with a task-tailored dexterous wrist enables robust rest-to-steady-state transitions.
  arXiv  Detail & Related papers  (2026-04-20T06:21:12Z)
• Design of an In-Pipe Robot with Contact-Angle-Guided Kinematic Decoupling for Crosstalk-Suppressed Locomotion [20.379309593600414]
  In-pipe inspection robots must traverse confined pipeline networks with elbows and three-dimensional fittings.<n>In compact V-shaped platforms, these functions often rely on shared contacts or indirect actuation.<n>This paper presents a V-shaped in-pipe robot with a joint-axis-and-wheel-separation layout that provides two physically independent actuation channels.
  arXiv  Detail & Related papers  (2026-03-28T11:40:36Z)
• Surface-Constrained Offline Warping with Contact-Aware Online Pose Projection for Safe Robotic Trajectory Execution [2.192782571038966]
  We present a two-stage framework that separates geometric embedding from execution-level regulation.<n>Experiments across multiple analytic surface families and real-robot validation on a sinusoidal surface demonstrate improved geometric continuity, reduced large orientation jumps, and robust contact maintenance.
  arXiv  Detail & Related papers  (2026-03-17T21:54:54Z)
• Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints [87.13154261503168]
  Motion-controllable video generation is crucial for egocentric applications in virtual reality and embodied AI.<n>Existing methods often struggle to achieve 3D-consistent fine-grained hand articulation.<n>We propose a novel framework that generates egocentric videos from a single reference frame.
  arXiv  Detail & Related papers  (2026-03-12T10:02:23Z)
• Structural Action Transformer for 3D Dexterous Manipulation [80.07649565189035]
  Cross-embodiment skill transfer is a challenge for high-DoF robotic hands.<n>Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations.<n>This paper proposes a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective.
  arXiv  Detail & Related papers  (2026-03-04T11:38:12Z)
• Robo3R: Enhancing Robotic Manipulation with Accurate Feed-Forward 3D Reconstruction [70.06600045165905]
  3D spatial perception is fundamental to general robotic manipulation, yet obtaining reliable, high-quality 3D geometry remains challenging.<n>We introduce Robo3R, a manipulation-ready 3D reconstruction model that predicts accurate, metric-scale scene geometry directly from RGB images and robot states in real time.<n>We observe consistent gains in performance, suggesting the promise of this alternative 3D sensing module for robotic manipulation.
  arXiv  Detail & Related papers  (2026-02-10T18:58:15Z)
• Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policies [32.99644520625179]
  This paper addresses the challenges of Rhythmic Insertion Tasks (RIT), where a robot must repeatedly perform high-precision insertions.<n>We propose a framework that integrates a reinforcement learning-based insertion policy with a failure forecasting module.<n>Our method achieves a high one-time success rate and robustly maintains performance over long-horizon repetitive tasks.
  arXiv  Detail & Related papers  (2025-07-09T03:38:44Z)
• AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch [9.606323817785114]
  We present AnyRotate, a system for gravity-invariant multi-axis in-hand object rotation using dense featured sim-to-real touch. Our formulation allows the training of a unified policy to rotate unseen objects about arbitrary rotation axes in any hand direction. Rich multi-fingered tactile sensing can detect unstable grasps and provide a reactive behavior that improves the robustness of the policy.
  arXiv  Detail & Related papers  (2024-05-12T22:51:35Z)
• NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation [53.25973084799954]
  We present NIKI (Neural Inverse Kinematics with Invertible Neural Network), which models bi-directional errors. NIKI can learn from both the forward and inverse processes with invertible networks.
  arXiv  Detail & Related papers  (2023-05-15T12:13:24Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.