DOT-Sim: Differentiable Optical Tactile Simulation with Precise Real-to-Sim Physical Calibration

• URL: http://arxiv.org/abs/2604.27367v1
• Date: Thu, 30 Apr 2026 03:27:09 GMT
• Title: DOT-Sim: Differentiable Optical Tactile Simulation with Precise Real-to-Sim Physical Calibration
• Abstract summary: We propose DOT-Sim: Differentiable Optical Tactile Simulation.<n> DOT-Sim accurately captures the physical behavior of soft sensors by modeling them as elastic materials.<n>To handle the optical aspect, we propose a novel approach to simulating optical responses by learning a residual image.
• Score: 12.64545897591664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Simulating optical tactile sensors presents significant challenges due to their high deformability and intricate optical properties. To address these issues and enable a physically accurate simulation, we propose DOT-Sim: Differentiable Optical Tactile Simulation. Unlike prior simulators that rely on simplified models of deformable sensors, DOT-Sim accurately captures the physical behavior of soft sensors by modeling them as elastic materials using the Material Point Method (MPM). DOT-Sim enables rapid calibration of optical tactile sensor simulation using a small number of demonstrations within minutes, which is substantially faster than existing methods. Compared to current baselines, our approach supports much larger and non-linear deformations. To handle the optical aspect, we propose a novel approach to simulating optical responses by learning a residual image relative to the real-world idle state. We validate the physical and visual realism of our method through a series of zero-shot sim-to-real tasks. Our experiments show that DOT-Sim (1) accurately replicates the physical dynamics of a DenseTact optical tactile sensor in reality, (2) generates realistic optical outputs in contact-rich scenarios, (3) enables direct deployment of simulation-trained classifiers in the real world, achieving 85% classification accuracy on challenging objects and 90% accuracy in embedded tumor-type detection, and (4) allows precise trajectory following with a policy trained from demonstrations in simulation, with an average error of less than 0.9 mm.

Related papers

• RealSimLoop: Online Real-to-Sim Adaptation via Differentiable Reduced-Order Simulation with Vision Feedback [20.409690218912512]
  We propose RealSimLoop, a differentiable framework for online real-to-sim adaptation using vision data as physical feedback.<n>Our approach achieves quasi-real-time performance by executing differentiable simulation within a reduced-order neural subspace.<n>By employing a sliding-window objective function, RealSimLoop enables robust online adaptation, allowing the system to track time-varying material properties.
  arXiv  Detail & Related papers  (2026-09-09T07:33:40Z)
• Tactile Sim2Real without Tactile Simulation via Bottlenecked Latent Reconstruction [11.374127908457446]
  Sim2Real avoids sensor-specific simulation entirely by training policies on a simulator-native oracle sensor.<n>We show that SBLR matches or approaches the performance of an oracle with direct access to tactile simulation.
  arXiv  Detail & Related papers  (2026-08-16T19:09:55Z)
• TactSpace: Learning a Physics-enriched Shared Latent Space for Tactile Sim-to-Real Transfer [8.357674069719524]
  We propose a multi-modal representation learning framework that aligns heterogeneous tactile modalities within a shared latent space.<n>Our approach employs modality-specific encoders to project diverse tactile observations into a common embedding space.<n>Our results demonstrate zero-shot sim-to-real transfer across physically dissimilar representations.
  arXiv  Detail & Related papers  (2026-06-17T11:41:27Z)
• SPLIT: Separating Physical-Contact via Latent Arithmetic in Image-Based Tactile Sensors [0.5500249707065662]
  This paper presents SPLIT, a novel method for simulating image-based tactile sensors.<n>Central to our approach is a latent space arithmetic strategy that explicitly disentangles contact geometry from sensor-specific optical properties.<n>We provide a calibrated finite element method (FEM) soft-body mesh simulation with variable resolution, offering a tunable trade-off between speed and fidelity.
  arXiv  Detail & Related papers  (2026-04-27T13:13:58Z)
• SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds [36.87909899472009]
  We introduce SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world.<n>The system digitizes scenes into metric-consistent twins, calibrates deformable dynamics through elastic modeling, and expands behaviors via diffusion-based trajectory generation.<n>Experiments show that policies trained on purely synthetic data achieve parity with real-data baselines at a 1:15 equivalence ratio, while delivering 90% zero-shot success and 50% generalization gains in real-world deployment.
  arXiv  Detail & Related papers  (2026-04-09T17:59:52Z)
• Resonance4D: Frequency-Domain Motion Supervision for Preset-Free Physical Parameter Learning in 4D Dynamic Physical Scene Simulation [53.141181758015186]
  We present Resonance4D, a physics-driven 4D dynamic simulation framework that couples 3D Gaussian Splatting with the Material Point Method.<n>Our key insight is that dynamic consistency can be enforced without dense temporal generation by jointly constraining motion in complementary domains.<n>We show that Resonance4D achieves strong physical fidelity and motion consistency while reducing peak GPU memory from over 35,GB to around 20,GB.
  arXiv  Detail & Related papers  (2026-04-02T13:00:22Z)
• Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling [9.084754731100311]
  We propose a few-shot real-to-sim approach that combines analytical formulations with the representational capacity of graph neural network (GNN)-based models.<n>Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions.<n> Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories.
  arXiv  Detail & Related papers  (2026-03-06T12:32:56Z)
• D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping [66.22412592525369]
  We introduce a real-to-sim-to-real engine that leverages the Gaussian Splat representations to build a differentiable engine.<n>We show that our engine achieves accurate and robust performance in mass identification across various object geometries and mass values.<n>Those optimized mass values facilitate force-aware policy learning, achieving superior and high performance in object grasping.
  arXiv  Detail & Related papers  (2026-03-01T15:32:04Z)
• End-to-end Differentiable Calibration and Reconstruction for Optical Particle Detectors [0.0]
  Analyses in experimental physics rely on high-fidelity simulators to translate sensor-level information into physical quantities of interest.<n>We present the first end-to-end differentiable optical particle detector simulator, enabling simultaneous calibration and reconstruction.<n>Our approach achieves simulation, calibration, and tracking, which are traditionally treated as separate problems, within a single differentiable framework.
  arXiv  Detail & Related papers  (2026-02-27T16:06:08Z)
• i-PhysGaussian: Implicit Physical Simulation for 3D Gaussian Splatting [60.46736489360263]
  i-PhysGaussian is a framework that couples 3D Gaussian Splatting (3DGS) with an implicit Material Point Method (MPM) integrator.<n>Unlike explicit methods, our solution obtains an end-of-step state by minimizing a momentum-balance residual.<n>Results demonstrate that i-PhysGaussian maintains stability at up to 20x larger time steps than explicit baselines.
  arXiv  Detail & Related papers  (2026-02-19T06:38:35Z)
• FastPhysGS: Accelerating Physics-based Dynamic 3DGS Simulation via Interior Completion and Adaptive Optimization [56.17833729527066]
  We propose FastPhysGS, a framework for physics-based dynamic 3DGS simulation.<n>FastPhysGS achieves high-fidelity physical simulation in 1 minute using only 7 GB runtime memory.
  arXiv  Detail & Related papers  (2026-02-02T07:00:42Z)
• GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects [55.02281855589641]
  GausSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.<n>We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter.<n>In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• Generic Lithography Modeling with Dual-band Optics-Inspired Neural Networks [52.200624127512874]
  We introduce a dual-band optics-inspired neural network design that considers the optical physics underlying lithography. Our approach yields the first published via/metal layer contour simulation at 1nm2/pixel resolution with any tile size. We also achieve 85X simulation speedup over traditional lithography simulator with 1% accuracy loss.
  arXiv  Detail & Related papers  (2022-03-12T08:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.