Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map

• URL: http://arxiv.org/abs/2602.21625v1
• Date: Wed, 25 Feb 2026 06:40:59 GMT
• Title: Tacmap: Bridging the Tactile Sim-to-Real Gap via Geometry-Consistent Penetration Depth Map
• Abstract summary: Vision-Based Tactile Sensors are essential for achieving dexterous robotic manipulation.<n>Current tactile simulations suffer from a persistent dilemma: simplified geometric projections lack physical authenticity.<n>We present Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in penetration depth.
• Score: 4.188018133636475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Based Tactile Sensors (VBTS) are essential for achieving dexterous robotic manipulation, yet the tactile sim-to-real gap remains a fundamental bottleneck. Current tactile simulations suffer from a persistent dilemma: simplified geometric projections lack physical authenticity, while high-fidelity Finite Element Methods (FEM) are too computationally prohibitive for large-scale reinforcement learning. In this work, we present Tacmap, a high-fidelity, computationally efficient tactile simulation framework anchored in volumetric penetration depth. Our key insight is to bridge the tactile sim-to-real gap by unifying both domains through a shared deform map representation. Specifically, we compute 3D intersection volumes as depth maps in simulation, while in the real world, we employ an automated data-collection rig to learn a robust mapping from raw tactile images to ground-truth depth maps. By aligning simulation and real-world in this unified geometric space, Tacmap minimizes domain shift while maintaining physical consistency. Quantitative evaluations across diverse contact scenarios demonstrate that Tacmap's deform maps closely mirror real-world measurements. Moreover, we validate the utility of Tacmap through an in-hand rotation task, where a policy trained exclusively in simulation achieves zero-shot transfer to a physical robot.

Related papers

• TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer [8.357674069719524]
  We propose a multi-modal representation learning framework that aligns heterogeneous tactile modalities within a shared latent space.<n>Our approach employs modality-specific encoders to project diverse tactile observations into a common embedding space.<n>Our results demonstrate zero-shot sim-to-real transfer across physically dissimilar representations.
  arXiv  Detail & Related papers  (2026-06-17T11:41:27Z)
• Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning [6.509572742369941]
  We propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand.<n>We introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals.<n>Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining.<n>Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy.
  arXiv  Detail & Related papers  (2026-06-10T07:46:38Z)
• Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation [51.463992799813816]
  A primary bottleneck in contact-rich manipulation is the difficulty of collecting real-world data.<n>We introduce Center-of-Pressure, an effective tactile representation grounded in physical principles.<n>We evaluate CoP on two blind, challenging contact-rich manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-27T17:59:02Z)
• TacLoc: Global Tactile Localization on Objects from a Registration Perspective [8.561002414977509]
  Pose estimation is essential for robotic manipulation, particularly when visual perception is occluded during gripper-object interactions.<n>Existing tactile-based methods rely on tactile simulation or pre-trained models, which limits their generalizability and efficiency.<n>We propose TacLoc, a novel tactile localization framework that formulates the problem as a one-shot point cloud registration task.
  arXiv  Detail & Related papers  (2026-03-11T09:15:07Z)
• Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation [82.63833405368159]
  Generalizing tool manipulation requires both semantic planning and precise physical control.<n>We propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates.<n> Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization.
  arXiv  Detail & Related papers  (2026-02-14T16:05:08Z)
• DeFM: Learning Foundation Representations from Depth for Robotics [49.77188649197404]
  We present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications.<n>DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors.<n>It achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments.
  arXiv  Detail & Related papers  (2026-01-26T19:45:31Z)
• SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation [63.48859753472547]
  SpatialActor is a framework for robust robotic manipulation that explicitly decouples semantics and geometry.<n>It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions.
  arXiv  Detail & Related papers  (2025-11-12T18:59:08Z)
• PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation [8.997347199266592]
  We introduce PseudoTouch which links high-dimensional structural information to low-dimensional sensor signals.<n>It does so by learning a low-dimensional visual-tactile embedding, wherein we encode a depth patch from which we decode the tactile signal.<n>We demonstrate the utility of our trained PseudoTouch model in two downstream tasks: object recognition and grasp stability prediction.
  arXiv  Detail & Related papers  (2024-03-22T10:51:31Z)
• Visual-Tactile Sensing for In-Hand Object Reconstruction [38.42487660352112]
  We propose a visual-tactile in-hand object reconstruction framework textbfVTacO, and extend it to textbfVTacOH for hand-object reconstruction. A simulation environment, VT-Sim, supports generating hand-object interaction for both rigid and deformable objects.
  arXiv  Detail & Related papers  (2023-03-25T15:16:31Z)
• Navigating to Objects in the Real World [76.1517654037993]
  We present a large-scale empirical study of semantic visual navigation methods comparing methods from classical, modular, and end-to-end learning approaches. We find that modular learning works well in the real world, attaining a 90% success rate. In contrast, end-to-end learning does not, dropping from 77% simulation to 23% real-world success rate due to a large image domain gap between simulation and reality.
  arXiv  Detail & Related papers  (2022-12-02T01:10:47Z)
• CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimization [54.69408516025872]
  CAP-UDF is a novel method to learn consistency-aware UDF from raw point clouds. We train a neural network to gradually infer the relationship between queries and the approximated surface. We also introduce a polygonization algorithm to extract surfaces using the gradients of the learned UDF.
  arXiv  Detail & Related papers  (2022-10-06T08:51:08Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Optical Tactile Sim-to-Real Policy Transfer via Real-to-Sim Tactile Image Translation [21.82940445333913]
  We present a suite of simulated environments tailored towards tactile robotics and reinforcement learning. A data-driven approach enables translation of the current state of a real tactile sensor to corresponding simulated depth images. This policy is implemented within a real-time control loop on a physical robot to demonstrate zero-shot sim-to-real policy transfer.
  arXiv  Detail & Related papers  (2021-06-16T13:58:35Z)
• Learning the sense of touch in simulation: a sim-to-real strategy for vision-based tactile sensing [1.9981375888949469]
  This paper focuses on a vision-based tactile sensor, which aims to reconstruct the distribution of the three-dimensional contact forces applied on its soft surface. A strategy is proposed to train a tailored deep neural network entirely from the simulation data. The resulting learning architecture is directly transferable across multiple tactile sensors without further training and yields accurate predictions on real data.
  arXiv  Detail & Related papers  (2020-03-05T14:17:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.