Cross-Sensor Touch Generation

• URL: http://arxiv.org/abs/2510.09817v1
• Date: Fri, 10 Oct 2025 19:32:15 GMT
• Title: Cross-Sensor Touch Generation
• Authors: Samanta Rodriguez, Yiming Dou, Miquel Oller, Andrew Owens, Nima Fazeli,
• Abstract summary: We propose two approaches to cross-sensor image generation.<n>The first is an end-to-end method that leverages paired data (Touch2Touch)<n>The second method builds an intermediate depth representation and does not require paired data (T2D2: Touch-to-Depth-to-Touch)
• Score: 31.58461878495434
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Today's visuo-tactile sensors come in many shapes and sizes, making it challenging to develop general-purpose tactile representations. This is because most models are tied to a specific sensor design. To address this challenge, we propose two approaches to cross-sensor image generation. The first is an end-to-end method that leverages paired data (Touch2Touch). The second method builds an intermediate depth representation and does not require paired data (T2D2: Touch-to-Depth-to-Touch). Both methods enable the use of sensor-specific models across multiple sensors via the cross-sensor touch generation process. Together, these models offer flexible solutions for sensor translation, depending on data availability and application needs. We demonstrate their effectiveness on downstream tasks such as in-hand pose estimation and behavior cloning, successfully transferring models trained on one sensor to another. Project page: https://samantabelen.github.io/cross_sensor_touch_generation.

Related papers

• Transferring Graph Neural Networks for Soft Sensor Modeling using Process Topologies [0.0]
  We propose a graph neural network approach for transfer learning of soft sensor models across multiple plants.<n>In our method, plants are modeled as graphs: Unit operations are nodes, streams are edges, and sensors are embedded as attributes.<n>We successfully transfer our soft sensor model to a previously unseen process with a different topology.
  arXiv  Detail & Related papers  (2025-02-05T12:10:22Z)
• MSSIDD: A Benchmark for Multi-Sensor Denoising [55.41612200877861]
  We introduce a new benchmark, the Multi-Sensor SIDD dataset, which is the first raw-domain dataset designed to evaluate the sensor transferability of denoising models. We propose a sensor consistency training framework that enables denoising models to learn the sensor-invariant features.
  arXiv  Detail & Related papers  (2024-11-18T13:32:59Z)
• ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception [1.5566524830295307]
  ACROSS is a framework for translating data between tactile sensors by exploiting sensor deformation information.<n>We transfer the tactile signals of a BioTac sensor to DIGIT tactile images.
  arXiv  Detail & Related papers  (2024-11-13T11:29:14Z)
• Transferring Tactile Data Across Sensors [1.5566524830295307]
  This article introduces a novel method for translating data between tactile sensors. We demonstrate the approach by translating BioTac signals into the DIGIT sensor. Our framework consists of three steps: first, converting signal data into corresponding 3D deformation meshes; second, translating these 3D deformation meshes from one sensor to another; and third, generating output images.
  arXiv  Detail & Related papers  (2024-10-18T09:15:47Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Binding Touch to Everything: Learning Unified Multimodal Tactile Representations [29.76008953177392]
  We introduce UniTouch, a unified model for vision-based touch sensors connected to multiple modalities. We achieve this by aligning our UniTouch embeddings to pretrained image embeddings already associated with a variety of other modalities. We further propose learnable sensor-specific tokens, allowing the model to learn from a set of heterogeneous tactile sensors.
  arXiv  Detail & Related papers  (2024-01-31T18:59:57Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• PyTouch: A Machine Learning Library for Touch Processing [68.32055581488557]
  We present PyTouch, the first machine learning library dedicated to the processing of touch sensing signals. PyTouch is designed to be modular, easy-to-use and provides state-of-the-art touch processing capabilities as a service. We evaluate PyTouch on real-world data from several tactile sensors on touch processing tasks such as touch detection, slip and object pose estimations.
  arXiv  Detail & Related papers  (2021-05-26T18:55:18Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)
• OmniTact: A Multi-Directional High Resolution Touch Sensor [109.28703530853542]
  Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. We introduce OmniTact, a multi-directional high-resolution tactile sensor. We evaluate the capabilities of OmniTact on a challenging robotic control task.
  arXiv  Detail & Related papers  (2020-03-16T01:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.