TactX: Learning Shared Tactile Representations Across Diverse Sensors

• URL: http://arxiv.org/abs/2606.31236v1
• Date: Tue, 30 Jun 2026 07:09:40 GMT
• Title: TactX: Learning Shared Tactile Representations Across Diverse Sensors
• Abstract summary: We propose TactX, a framework for learning a transferable tactile representation across sensors.<n>TactX maps heterogeneous tactile observations into a shared latent space through modality-specific encoders trained on paired contact data.<n>Experiments show that TactX aligns tactile representations across sensors while preserving object-level contact information.
• Score: 18.127088232068456
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tactile sensors provide critical information for contact-rich manipulation, yet tactile representations and policies remain tightly coupled to each specific sensor, limiting transferability across robots and hardware platforms. We propose TactX, a framework for learning a transferable tactile representation across sensors spanning three fundamentally different transduction modalities: resistive, magnetic, and vision-based. TactX maps heterogeneous tactile observations into a shared latent space through modality-specific encoders trained on paired contact data. Such paired interactions provide a natural alignment signal across modalities, and the encoders are jointly trained across all sensor pairs, inducing a consistent latent space for all sensor types. Our experiments show that TactX aligns tactile representations across sensors while preserving object-level contact information, as evidenced by sensor-identity prediction and object classification in the learned latent space. We evaluate TactX on four contact-rich manipulation tasks: pick-and-place, plug insertion, board wiping, and object reorientation, and show that policies trained with one sensor transfer zero-shot to physically distinct sensors through the shared latent. This improves the average success rate from 27.5% for vision-only policy to 45.9%, providing a step toward sensor-agnostic tactile manipulation.

Related papers

• Heterogeneous Tactile Transformer [12.676986440348543]
  Heterogeneous Tactile Transformer (HTT) is a framework that learns shared tactile representations across heterogeneous sensors.<n>Pretraining uses our novel Heterogeneous Paired Tactile dataset, containing 1.6M synchronized frames across four vision- and array-based tactile sensors.<n> HTT is shown to learn transferable representations that adapt to new tasks and previously unseen sensors.
  arXiv  Detail & Related papers  (2026-06-29T08:24:38Z)
• TacVerse: A Multi-Sensor Dataset and Benchmark for Cross-Sensor Vision-Based Tactile Perception [7.968185063285978]
  We present TacVerse, a multi-sensor dataset and benchmark for cross-sensor vision-based tactile perception.<n>The dataset contains 106,800 tactile images from seven vision-based tactile sensors.<n>Strong within-sensor performance across all tasks indicates that the collected tactile observations are informative for the target objectives.
  arXiv  Detail & Related papers  (2026-06-24T14:22:48Z)
• FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation [66.22092641373067]
  FTP-1 is the first generalist foundation tactile policy pretrained to acquire transferable tactile manipulation abilities.<n>It supports varied tactile inputs, including image-, array-, and state-based signals.<n>It learns tactile skills that transfer beyond the sensors seen during pretraining.
  arXiv  Detail & Related papers  (2026-06-11T09:30:09Z)
• Multi-Resolution Tactile Imitation Learning for Contact-Rich Robotic Manipulation [22.015203192700586]
  We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions.<n>MiTaS achieves an average success rate of 80 %, while vision-only (31 and visual-tactile (54 baselines cannot solve the task reliably)<n>A detailed sensor-reading and attention analysis reveals the importance of different sensors throughout task execution.
  arXiv  Detail & Related papers  (2026-06-04T15:23:03Z)
• TacO: Benchmarking Tactile Sensors for Object Manipulation [19.422699530305966]
  We provide a systematic, task-driven evaluation of tactile sensors for robot manipulation.<n>We propose a framework for selecting and evaluating sensors based on manipulation policy performance.<n>All of the tactile sensors, code, data, and hardware setup will be publicly available on the project website.
  arXiv  Detail & Related papers  (2026-05-21T04:11:03Z)
• UniForce: A Unified Latent Force Model for Robot Manipulation with Diverse Tactile Sensors [51.88112610411651]
  We propose a novel unified tactile representation learning framework that learns a shared latent force space across diverse tactile sensors.<n>UniForce reduces cross-sensor domain shift by jointly modeling inverse dynamics (image-to-force) and forward dynamics (force-to-image)<n>To avoid reliance on expensive external force/torque (F/T) sensors, we exploit static equilibrium and collect force-paired data via direct sensor--object--sensor interactions.
  arXiv  Detail & Related papers  (2026-02-01T11:03:01Z)
• OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction [93.88239833545623]
  We present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset.<n>We show that tactile signals provide a compact yet powerful cue for grasp understanding.<n>We aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation.
  arXiv  Detail & Related papers  (2025-12-18T18:18:17Z)
• AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors [11.506370451126378]
  Visuo-tactile sensors aim to emulate human tactile perception, enabling robots to understand and manipulate objects.<n>We introduce TacQuad, an aligned multi-modal tactile multi-sensor dataset from four different visuo-tactile sensors.<n>We propose AnyTouch, a unified static-dynamic multi-sensor representation learning framework with a multi-level structure.
  arXiv  Detail & Related papers  (2025-02-15T08:33:25Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)
• OmniTact: A Multi-Directional High Resolution Touch Sensor [109.28703530853542]
  Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. We introduce OmniTact, a multi-directional high-resolution tactile sensor. We evaluate the capabilities of OmniTact on a challenging robotic control task.
  arXiv  Detail & Related papers  (2020-03-16T01:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.