DexTac: Learning Contact-aware Visuotactile Policies via Hand-by-hand Teaching

• URL: http://arxiv.org/abs/2601.21474v1
• Date: Thu, 29 Jan 2026 09:52:46 GMT
• Title: DexTac: Learning Contact-aware Visuotactile Policies via Hand-by-hand Teaching
• Abstract summary: We present DexTac, a visuo-tactile manipulation learning framework based on kinesthetic teaching.<n>We evaluate our framework on a challenging unimanual injection task.
• Score: 14.892858615291717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For contact-intensive tasks, the ability to generate policies that produce comprehensive tactile-aware motions is essential. However, existing data collection and skill learning systems for dexterous manipulation often suffer from low-dimensional tactile information. To address this limitation, we propose DexTac, a visuo-tactile manipulation learning framework based on kinesthetic teaching. DexTac captures multi-dimensional tactile data-including contact force distributions and spatial contact regions-directly from human demonstrations. By integrating these rich tactile modalities into a policy network, the resulting contact-aware agent enables a dexterous hand to autonomously select and maintain optimal contact regions during complex interactions. We evaluate our framework on a challenging unimanual injection task. Experimental results demonstrate that DexTac achieves a 91.67% success rate. Notably, in high-precision scenarios involving small-scale syringes, our approach outperforms force-only baselines by 31.67%. These results underscore that learning multi-dimensional tactile priors from human demonstrations is critical for achieving robust, human-like dexterous manipulation in contact-rich environments.

Related papers

• Dex-X: Learning Visual-Tactile Dexterous Manipulation From Human Videos with Simulated Interaction [30.46515328565822]
  We present DEX-X, a framework for learning visual-tactile dexterous manipulation from human videos through simulation.<n>We reconstruct hand-object interactions in simulation, where physically grounded contact dynamics provide tactile supervision unavailable in the original videos.<n>We demonstrate zero-shot sim-to-real transfer on a dexterous hand-arm platform across diverse grasping and contact-rich tool-use tasks.
  arXiv  Detail & Related papers  (2026-09-07T16:47:39Z)
• T-Rex: Tactile-Reactive Dexterous Manipulation [139.71263755530654]
  We introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder.<n>We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation.
  arXiv  Detail & Related papers  (2026-06-15T17:59:55Z)
• DexViTac: Collecting Human Visuo-Tactile-Kinematic Demonstrations for Contact-Rich Dexterous Manipulation [4.463599051210204]
  Large-scale, high-quality multimodal demonstrations are essential for robot learning of contact-rich dexterous manipulation.<n>We present DexViTac, a portable, human-centric data collection system tailored for contact-rich dexterous manipulation.
  arXiv  Detail & Related papers  (2026-03-18T15:39:58Z)
• Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLA [62.16042475700567]
  We introduce IMCopilot, a shared-autonomy assistant to simplify teleoperation data collection.<n>We present MoDE-VLA, an architecture that seamlessly integrates heterogeneous force and tactile modalities into a pretrained VLA backbone.<n>We validate our approach on four tasks of escalating complexity, demonstrating doubled success rate improvement over the baseline in dexterous contact-rich tasks.
  arXiv  Detail & Related papers  (2026-03-09T09:02:30Z)
• TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance [53.35296919674763]
  TouchGuide is a novel cross-policy visuo-tactile fusion paradigm that fuses modalities within a low-dimensional action space.<n>touchGuide operates in two stages to guide a pre-trained diffusion or flow-matching visuomotor policy.<n>To facilitate TouchGuide training with high-quality and cost-effective data, we introduce TacUMI, a data collection system.
  arXiv  Detail & Related papers  (2026-01-28T04:22:47Z)
• DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation [25.208854363099352]
  This dataset contains 7,000 hours of dexterous hand-object interactions seeded from 70 hours of real human demonstrations.<n>Each entry combines synchronized multi-view RGB-D, high-precision mocap with MANO hand parameters, and per-frame contact points with physically consistent force profiles.<n>Our real-to-sim pipeline uses reinforcement learning to train policies that control an actuated MANO hand in physics simulation.
  arXiv  Detail & Related papers  (2025-10-17T16:08:14Z)
• Dexterous Manipulation through Imitation Learning: A Survey [32.47862766672596]
  Imitation learning (IL) offers an alternative by allowing robots to acquire dexterous manipulation skills directly from expert demonstrations.<n>IL captures fine-grained coordination and contact dynamics while bypassing the need for explicit modeling and large-scale trial-and-error.<n>Our goal is to offer researchers and practitioners a comprehensive introduction to this rapidly evolving domain.
  arXiv  Detail & Related papers  (2025-04-04T15:14:38Z)
• DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulation [78.60543357822957]
  Dexterous manipulation with contact-rich interactions is crucial for advanced robotics.<n>We introduce DexHandDiff, an interaction-aware diffusion planning framework for adaptive dexterous manipulation.<n>Our framework achieves an average of 70.7% success rate on goal adaptive dexterous tasks, highlighting its robustness and flexibility in contact-rich manipulation.
  arXiv  Detail & Related papers  (2024-11-27T18:03:26Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play [15.780086627089885]
  T-Dex is a new approach for tactile-based dexterity that operates in two phases. In the first phase, we collect 2.5 hours of play data, which is used to train self-supervised tactile encoders. In the second phase, given a handful of demonstrations for a dexterous task, we learn non-parametric policies that combine the tactile observations with visual ones.
  arXiv  Detail & Related papers  (2023-03-21T17:59:20Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.