Semi-Supervised Disentanglement of Tactile Contact~Geometry from Sliding-Induced Shear

• URL: http://arxiv.org/abs/2208.12500v1
• Date: Fri, 26 Aug 2022 08:30:19 GMT
• Title: Semi-Supervised Disentanglement of Tactile Contact~Geometry from Sliding-Induced Shear
• Abstract summary: The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. In this work, we pursue a semi-supervised approach to remove shear while preserving contact-only information.
• Score: 12.004939546183355
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The sense of touch is fundamental to human dexterity. When mimicked in robotic touch, particularly by use of soft optical tactile sensors, it suffers from distortion due to motion-dependent shear. This complicates tactile tasks like shape reconstruction and exploration that require information about contact geometry. In this work, we pursue a semi-supervised approach to remove shear while preserving contact-only information. We validate our approach by showing a match between the model-generated unsheared images with their counterparts from vertically tapping onto the object. The model-generated unsheared images give faithful reconstruction of contact-geometry otherwise masked by shear, along with robust estimation of object pose then used for sliding exploration and full reconstruction of several planar shapes. We show that our semi-supervised approach achieves comparable performance to its fully supervised counterpart across all validation tasks with an order of magnitude less supervision. The semi-supervised method is thus more computational and labeled sample-efficient. We expect it will have broad applicability to wide range of complex tactile exploration and manipulation tasks performed via a shear-sensitive sense of touch.

Related papers

• AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion [60.79507611737292]
  Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback.<n>Existing robotic grasping approaches rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact.<n>We propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement.
  arXiv  Detail & Related papers  (2026-08-06T16:29:11Z)
• Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation [74.87716678195099]
  Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation.<n>Despite their advantages, extracting fine-grained contact states necessary for contact-rich manipulation remains an open challenge.<n>We propose a motion-aware tactile representation to facilitate contact-rich manipulation.
  arXiv  Detail & Related papers  (2026-06-29T08:20:18Z)
• Geometric Reconstruction of Extrinsic Contact Trajectories using Tactile Sensing and Proprioception for Tool Manipulation [3.853357303181037]
  In tool-mediated manipulation tasks, the interaction that determines task success occurs at the tool tip, away from the tactile sensor.<n>In this work, we reconstruct the trajectory of extrinsic tool-tip contact using tactile sensing and robot proprioception.
  arXiv  Detail & Related papers  (2026-06-20T22:28:16Z)
• Physics-Informed Eikonal Caging for Whole-Arm Manipulation Planning [29.754938212556223]
  We introduce caging as a minimum-time escape problem in which the object seeks to leave an enclosing robot geometry in the shortest time.<n>This yields a continuous escape-time field that measures the robot's enclosure quality and we show it satisfies an eikonal equation.<n>We therefore can approximate this field using a physics-informed neural network, producing a smooth differentiable representation that can be embedded directly into manipulation planning.
  arXiv  Detail & Related papers  (2026-06-20T16:49:17Z)
• Touch2Insert: Zero-Shot Peg Insertion by Touching Intersections of Peg and Hole [15.791523980428538]
  We present emphTouch2Insert, a tactile-based framework for arbitrary peg insertion.<n>Our method reconstructs cross-sectional geometry from high-resolution tactile images and estimates the relative pose of the hole with respect to the peg in a zero-shot manner.<n>Results indicate that Touch2Insert achieved sub-millimeter pose estimation accuracy for all connectors in simulation, and attained an average success rate of 86.7% on the real robot.
  arXiv  Detail & Related papers  (2026-03-04T01:35:33Z)
• MeshMimic: Geometry-Aware Humanoid Motion Learning through 3D Scene Reconstruction [54.36564144414704]
  MeshMimic is an innovative framework that bridges 3D scene reconstruction and embodied intelligence to enable humanoid robots to learn coupled "motion-terrain" interactions directly from video.<n>By leveraging state-of-the-art 3D vision models, our framework precisely segments and reconstructs both human trajectories and the underlying 3D geometry of terrains and objects.
  arXiv  Detail & Related papers  (2026-02-17T17:09:45Z)
• Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation [82.63833405368159]
  Generalizing tool manipulation requires both semantic planning and precise physical control.<n>We propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates.<n> Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization.
  arXiv  Detail & Related papers  (2026-02-14T16:05:08Z)
• SpatialActor: Exploring Disentangled Spatial Representations for Robust Robotic Manipulation [63.48859753472547]
  SpatialActor is a framework for robust robotic manipulation that explicitly decouples semantics and geometry.<n>It achieves state-of-the-art performance with 87.4% on RLBench and improves by 13.9% to 19.4% under varying noisy conditions.
  arXiv  Detail & Related papers  (2025-11-12T18:59:08Z)
• Object Pose Estimation through Dexterous Touch [27.99244228962149]
  Our approach uses sensorimotor exploration to actively control a robot hand to interact with the object.<n>We show that our method can actively explore an object's surface to identify critical pose features without prior knowledge of the object's geometry.
  arXiv  Detail & Related papers  (2025-09-16T23:25:05Z)
• ViTaSCOPE: Visuo-tactile Implicit Representation for In-hand Pose and Extrinsic Contact Estimation [2.140861702387444]
  dexterous contact-rich object manipulation demands both object poses and external contact locations.<n>We present ViTaSCOPE: VisuoTac Simultaneous Contact and Object Estimation.
  arXiv  Detail & Related papers  (2025-06-13T21:35:58Z)
• Towards Generalization of Tactile Image Generation: Reference-Free Evaluation in a Leakage-Free Setting [25.355424080824996]
  Tactile sensing is critical for human perception and underpins applications in computer vision, robotics, and multimodal learning. Because tactile data is often scarce and costly to acquire, generating synthetic tactile images provides a scalable solution to augment real-world measurements. We demonstrate that overlapping training and test samples in commonly used datasets inflate performance metrics, obscuring the true generalizability of tactile models.
  arXiv  Detail & Related papers  (2025-03-10T02:37:22Z)
• GEARS: Local Geometry-aware Hand-object Interaction Synthesis [38.75942505771009]
  We introduce a novel joint-centered sensor designed to reason about local object geometry near potential interaction regions. As an important step towards mitigating the learning complexity, we transform the points from global frame to template hand frame and use a shared module to process sensor features of each individual joint. This is followed by a perceptual-temporal transformer network aimed at capturing correlation among the joints in different dimensions.
  arXiv  Detail & Related papers  (2024-04-02T09:18:52Z)
• Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces [34.831730064258494]
  We propose Tactile-Informed 3DGS, a novel approach that incorporates touch data (local depth maps) with multi-view vision data to achieve surface reconstruction and novel view synthesis. By creating a framework that decreases the transmittance at touch locations, we achieve a refined surface reconstruction, ensuring a uniformly smooth depth map. We conduct evaluation on objects with glossy and reflective surfaces and demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-03-29T16:30:17Z)
• Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Images [59.49985837246644]
  We show how to model contacts in an explicit way to benefit the implicit reconstruction of hand-held objects. In the first part, we propose a new subtask of directly estimating 3D hand-object contacts from a single image. In the second part, we introduce a novel method to diffuse estimated contact states from the hand mesh surface to nearby 3D space.
  arXiv  Detail & Related papers  (2023-05-31T17:59:26Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch [9.31776719215139]
  We propose an approach to grasp planning that explicitly reasons about where the fingertips should contact the estimated object surface. Key to our method's success is the use of visual surface estimation for initial planning to encode the contact constraint. We show that our method successfully synthesises and executes precision grasps for previously unseen objects using surface estimates from a single camera view.
  arXiv  Detail & Related papers  (2022-12-16T17:32:56Z)
• Learning to Detect Slip with Barometric Tactile Sensors and a Temporal Convolutional Neural Network [7.346580429118843]
  We present a learning-based method to detect slip using barometric tactile sensors. We train a temporal convolution neural network to detect slip, achieving high detection accuracies. We argue that barometric tactile sensing technology, combined with data-driven learning, is suitable for many manipulation tasks such as slip compensation.
  arXiv  Detail & Related papers  (2022-02-19T08:21:56Z)
• Contact-Aware Retargeting of Skinned Motion [49.71236739408685]
  This paper introduces a motion estimation method that preserves self-contacts and prevents interpenetration. The method identifies self-contacts and ground contacts in the input motion, and optimize the motion to apply to the output skeleton. In experiments, our results quantitatively outperform previous methods and we conduct a user study where our retargeted motions are rated as higher-quality than those produced by recent works.
  arXiv  Detail & Related papers  (2021-09-15T17:05:02Z)
• Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear [30.404840177562754]
  Robotic touch, particularly when using soft optical tactile sensors, suffers from distortion caused by motion-dependent shear. We propose a supervised convolutional deep neural network model that learns to disentangle, in the latent space, the components of sensor deformations caused by contact geometry from those due to sliding-induced shear.
  arXiv  Detail & Related papers  (2021-09-08T13:03:08Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Active 3D Shape Reconstruction from Vision and Touch [66.08432412497443]
  Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. In 3D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings. We introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile priors to guide the shape exploration; and 3) a set of data-driven solutions with either tactile or visuo
  arXiv  Detail & Related papers  (2021-07-20T15:56:52Z)
• 3D Shape Reconstruction from Vision and Touch [62.59044232597045]
  In 3D shape reconstruction, the complementary fusion of visual and haptic modalities remains largely unexplored. We introduce a dataset of simulated touch and vision signals from the interaction between a robotic hand and a large array of 3D objects.
  arXiv  Detail & Related papers  (2020-07-07T20:20:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.