TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks

• URL: http://arxiv.org/abs/2604.07335v1
• Date: Wed, 08 Apr 2026 17:49:35 GMT
• Title: TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks
• Abstract summary: TAMEn is a tactile-aware manipulation engine for closed-loop data collection in contact-rich tasks.<n>Our system features a cross-morphology wearable interface that enables rapid adaptation across heterogeneous grippers.<n>Our proposed visuo-tactile learning framework increases task success rates from 34% to 75% across diverse bimanual manipulation tasks.
• Score: 16.262130849751312
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Handheld paradigms offer an efficient and intuitive way for collecting large-scale demonstration of robot manipulation. However, achieving contact-rich bimanual manipulation through these methods remains a pivotal challenge, which is substantially hindered by hardware adaptability and data efficacy. Prior hardware designs remain gripper-specific and often face a trade-off between tracking precision and portability. Furthermore, the lack of online feasibility checking during demonstration leads to poor replayability. More importantly, existing handheld setups struggle to collect interactive recovery data during robot execution, lacking the authentic tactile information necessary for robust policy refinement. To bridge these gaps, we present TAMEn, a tactile-aware manipulation engine for closed-loop data collection in contact-rich tasks. Our system features a cross-morphology wearable interface that enables rapid adaptation across heterogeneous grippers. To balance data quality and environmental diversity, we implement a dual-modal acquisition pipeline: a precision mode leveraging motion capture for high-fidelity demonstrations, and a portable mode utilizing VR-based tracking for in-the-wild acquisition and tactile-visualized recovery teleoperation. Building on this hardware, we unify large-scale tactile pretraining, task-specific bimanual demonstrations, and human-in-the-loop recovery data into a pyramid-structured data regime, enabling closed-loop policy refinement. Experiments show that our feasibility-aware pipeline significantly improves demonstration replayability, and that the proposed visuo-tactile learning framework increases task success rates from 34% to 75% across diverse bimanual manipulation tasks. We further open-source the hardware and dataset to facilitate reproducibility and support research in visuo-tactile manipulation.

Related papers

• HapTile: A Haptic-Informed Vision-Tactile-Language-Action Dataset for Contact-Rich Imitation Learning [24.64460457808404]
  HapTile is a contact-grounded visuotactile manipulation dataset that advances beyond vision-only trajectory datasets.<n>The data collection platform integrates haptic feedback directly into the teleoperation controller, enabling the operator to perceive contact interactions in real time.<n>The dataset comprises everyday manipulation tasks spanning a broad range of contact-rich skills, including pick-and-place, folding, pressing, stacking, and other routine activities.
  arXiv  Detail & Related papers  (2026-06-03T12:48:17Z)
• VTouch++: A Multimodal Dataset with Vision-Based Tactile Enhancement for Bimanual Manipulation [10.08235353271524]
  Embodied intelligence has advanced rapidly in recent years; however, bimanual manipulation-especially in contact-rich tasks remains challenging.<n>This is largely due to the lack of datasets with rich physical interaction signals, systematic task organization, and sufficient scale.<n>We introduce the VTOUCH dataset to provide high-fidelity physical interaction signals, adopt a matrix-style task design, and employ automated data collection pipelines covering real-world, demand-driven scenarios.
  arXiv  Detail & Related papers  (2026-04-22T11:08:08Z)
• OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation [57.133721026727706]
  We present textbfOmniViTac, a large-scale visuo-tactile-action dataset comprising $21,000+$ trajectories across $86$ tasks and $100+$ objects.<n>We propose textbf OmniVTA, a world-model-based visuo-tactile manipulation framework that integrates four tightly coupled modules.
  arXiv  Detail & Related papers  (2026-03-19T17:52:42Z)
• DexViTac: Collecting Human Visuo-Tactile-Kinematic Demonstrations for Contact-Rich Dexterous Manipulation [4.463599051210204]
  Large-scale, high-quality multimodal demonstrations are essential for robot learning of contact-rich dexterous manipulation.<n>We present DexViTac, a portable, human-centric data collection system tailored for contact-rich dexterous manipulation.
  arXiv  Detail & Related papers  (2026-03-18T15:39:58Z)
• TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance [53.35296919674763]
  TouchGuide is a novel cross-policy visuo-tactile fusion paradigm that fuses modalities within a low-dimensional action space.<n>touchGuide operates in two stages to guide a pre-trained diffusion or flow-matching visuomotor policy.<n>To facilitate TouchGuide training with high-quality and cost-effective data, we introduce TacUMI, a data collection system.
  arXiv  Detail & Related papers  (2026-01-28T04:22:47Z)
• Tactile Memory with Soft Robot: Robust Object Insertion via Masked Encoding and Soft Wrist [10.982180941605256]
  We introduce Tactile Memory with Soft Robot (TaSo-bot), a system that integrates a soft wrist with retrieval-based control to enable safe and robust manipulation.<n>The core of this system is the Masked Tactile Trajectory Transformer (MATtext3$), which jointly models interactions between robot actions, tactile feedback, force-torque measurements, and proprioceptive signals.<n>MATtext3$ achieves higher success rates than the baselines over all conditions and shows remarkable capability to adapt to unseen pegs and conditions.
  arXiv  Detail & Related papers  (2026-01-27T07:04:01Z)
• Learning Generalizable Hand-Object Tracking from Synthetic Demonstrations [71.15185260689928]
  We present a system for learning generalizable hand-object tracking controllers purely from synthetic data, without requiring any human demonstrations.<n>Our approach makes two key contributions: (1) HOP, a Hand-Object Planner, which can synthesize diverse hand-object trajectories; and (2) HOT, a Hand-Object Tracker, that bridges synthetic-to-physical transfer through reinforcement learning and interaction imitation learning.
  arXiv  Detail & Related papers  (2025-12-22T17:08:54Z)
• Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper [7.618517580705364]
  We present a portable, lightweight gripper with integrated tactile sensors.<n>We propose a cross-modal representation learning framework that integrates visual and tactile signals.<n>We validate our approach on fine-grained tasks such as test tube insertion and pipette-based fluid transfer.
  arXiv  Detail & Related papers  (2025-07-20T17:53:59Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Multi-dataset Training of Transformers for Robust Action Recognition [75.5695991766902]
  We study the task of robust feature representations, aiming to generalize well on multiple datasets for action recognition. Here, we propose a novel multi-dataset training paradigm, MultiTrain, with the design of two new loss terms, namely informative loss and projection loss. We verify the effectiveness of our method on five challenging datasets, Kinetics-400, Kinetics-700, Moments-in-Time, Activitynet and Something-something-v2.
  arXiv  Detail & Related papers  (2022-09-26T01:30:43Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.