Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear

• URL: http://arxiv.org/abs/2109.03615v1
• Date: Wed, 8 Sep 2021 13:03:08 GMT
• Title: Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear
• Abstract summary: Robotic touch, particularly when using soft optical tactile sensors, suffers from distortion caused by motion-dependent shear. We propose a supervised convolutional deep neural network model that learns to disentangle, in the latent space, the components of sensor deformations caused by contact geometry from those due to sliding-induced shear.
• Score: 30.404840177562754
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Robotic touch, particularly when using soft optical tactile sensors, suffers from distortion caused by motion-dependent shear. The manner in which the sensor contacts a stimulus is entangled with the tactile information about the geometry of the stimulus. In this work, we propose a supervised convolutional deep neural network model that learns to disentangle, in the latent space, the components of sensor deformations caused by contact geometry from those due to sliding-induced shear. The approach is validated by reconstructing unsheared tactile images from sheared images and showing they match unsheared tactile images collected with no sliding motion. In addition, the unsheared tactile images give a faithful reconstruction of the contact geometry that is not possible from the sheared data, and robust estimation of the contact pose that can be used for servo control sliding around various 2D shapes. Finally, the contact geometry reconstruction in conjunction with servo control sliding were used for faithful full object reconstruction of various 2D shapes. The methods have broad applicability to deep learning models for robots with a shear-sensitive sense of touch.

Related papers

• Proximity3D: Shape from Capacitive Proximity on Sensing Manifold [13.523536411772314]
  We use a curved capacitive textile as a shape sensor, treating its surface as a non-planar sensing manifold.<n>We introduce a multi-view feedforward reconstruction model that aggregates these fields across known sensor views and recovers the observed object shape.
  arXiv  Detail & Related papers  (2026-08-31T07:00:19Z)
• AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion [60.79507611737292]
  Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback.<n>Existing robotic grasping approaches rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact.<n>We propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement.
  arXiv  Detail & Related papers  (2026-08-06T16:29:11Z)
• Toward Geometry-Scalable Whole-Body Touch for Humanoids: A 3D-Printed Conformal EIT Skin [66.36050076244905]
  Whole-body tactile sensing is a prerequisite for humanoids that operate in contact-rich human environments.<n>We present a conformal electrical tactile tomography skin fabricated through a geometry-adaptable additive-manufacturing workflow.
  arXiv  Detail & Related papers  (2026-08-03T11:33:30Z)
• Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation [74.87716678195099]
  Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation.<n>Despite their advantages, extracting fine-grained contact states necessary for contact-rich manipulation remains an open challenge.<n>We propose a motion-aware tactile representation to facilitate contact-rich manipulation.
  arXiv  Detail & Related papers  (2026-06-29T08:20:18Z)
• Geometric Reconstruction of Extrinsic Contact Trajectories using Tactile Sensing and Proprioception for Tool Manipulation [3.853357303181037]
  In tool-mediated manipulation tasks, the interaction that determines task success occurs at the tool tip, away from the tactile sensor.<n>In this work, we reconstruct the trajectory of extrinsic tool-tip contact using tactile sensing and robot proprioception.
  arXiv  Detail & Related papers  (2026-06-20T22:28:16Z)
• CHOIR: Contact-aware 4D Hand-Object Interaction Reconstruction [73.97421249653412]
  We present CHOIR, a Contact-aware HOI Reconstruction framework for a monocular camera.<n>ChoIR first initializes a coarse, contact-agnostic 4D HOI sequence from open-world visual priors.<n>It then introduces a generative HOI spatial rectification module to predict ray-depth corrections and rectify hand-object relative placement.<n>Last, a contact-aware joint optimization with dynamically updated contact constraints enforces geometric, temporal, and contact consistency.
  arXiv  Detail & Related papers  (2026-05-20T10:31:10Z)
• TouchAnything: Diffusion-Guided 3D Reconstruction from Sparse Robot Touches [18.3533066960967]
  We present TouchAnything, a framework that leverages a pretrained vision diffusion model as a semantic and geometric prior for 3D reconstruction from sparse tactile measurements.<n>Our method reconstructs accurate geometries from only a few touches, outperforms existing baselines, and enables open-world 3D reconstruction of previously unseen object instances.
  arXiv  Detail & Related papers  (2026-04-10T04:26:36Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• GRACE: Estimating Geometry-level 3D Human-Scene Contact from 2D Images [54.602947113980655]
  Estimating the geometry level of human-scene contact aims to ground specific contact surface points at 3D human geometries.<n> GRACE (Geometry-level Reasoning for 3D Human-scene Contact Estimation) is a new paradigm for 3D human contact estimation.<n>It incorporates a point cloud encoder-decoder architecture along with a hierarchical feature extraction and fusion module.
  arXiv  Detail & Related papers  (2025-05-10T09:25:46Z)
• TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing [29.691786688595762]
  Humans rely on their visual and tactile senses to develop a comprehensive 3D understanding of their physical environment. We propose TouchSDF, a Deep Learning approach for tactile 3D shape reconstruction. Our technique consists of two components: (1) a Convolutional Neural Network that maps tactile images into local meshes representing the surface at the touch location, and (2) an implicit neural function that predicts a signed distance function to extract the desired 3D shape.
  arXiv  Detail & Related papers  (2023-11-21T13:43:06Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Images [59.49985837246644]
  We show how to model contacts in an explicit way to benefit the implicit reconstruction of hand-held objects. In the first part, we propose a new subtask of directly estimating 3D hand-object contacts from a single image. In the second part, we introduce a novel method to diffuse estimated contact states from the hand mesh surface to nearby 3D space.
  arXiv  Detail & Related papers  (2023-05-31T17:59:26Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Semi-Supervised Disentanglement of Tactile Contact~Geometry from Sliding-Induced Shear [12.004939546183355]
  The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. In this work, we pursue a semi-supervised approach to remove shear while preserving contact-only information.
  arXiv  Detail & Related papers  (2022-08-26T08:30:19Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Active 3D Shape Reconstruction from Vision and Touch [66.08432412497443]
  Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. In 3D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings. We introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile priors to guide the shape exploration; and 3) a set of data-driven solutions with either tactile or visuo
  arXiv  Detail & Related papers  (2021-07-20T15:56:52Z)
• GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger [8.047951969722794]
  GelSight Wedge sensor is optimized to have a compact shape for robot fingers, while achieving high-resolution 3D reconstruction. We show the effectiveness and potential of the reconstructed 3D geometry for pose tracking in the 3D space.
  arXiv  Detail & Related papers  (2021-06-16T15:15:29Z)
• Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering [19.69677059281393]
  We present an approach to tactile pose estimation from the first touch for known objects. We create an object-agnostic map from real tactile observations to contact shapes. For a new object with known geometry, we learn a tailored perception model completely in simulation.
  arXiv  Detail & Related papers  (2020-12-09T18:00:35Z)
• OmniTact: A Multi-Directional High Resolution Touch Sensor [109.28703530853542]
  Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. We introduce OmniTact, a multi-directional high-resolution tactile sensor. We evaluate the capabilities of OmniTact on a challenging robotic control task.
  arXiv  Detail & Related papers  (2020-03-16T01:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.