Simultaneous Extrinsic Contact and In-Hand Pose Estimation via Distributed Tactile Sensing

• URL: http://arxiv.org/abs/2512.23856v1
• Date: Mon, 29 Dec 2025 20:45:38 GMT
• Title: Simultaneous Extrinsic Contact and In-Hand Pose Estimation via Distributed Tactile Sensing
• Abstract summary: We propose a method that pairs local observations from sensing with the physical constraints of contact.<n>We formalize our problem as a factor graph, allowing for efficient estimation.<n>In our experiments, we demonstrate that our method outperforms existing geometric and contact-informed estimation pipelines.
• Score: 20.162368645967153
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prehensile autonomous manipulation, such as peg insertion, tool use, or assembly, require precise in-hand understanding of the object pose and the extrinsic contacts made during interactions. Providing accurate estimation of pose and contacts is challenging. Tactile sensors can provide local geometry at the sensor and force information about the grasp, but the locality of sensing means resolving poses and contacts from tactile alone is often an ill-posed problem, as multiple configurations can be consistent with the observations. Adding visual feedback can help resolve ambiguities, but can suffer from noise and occlusions. In this work, we propose a method that pairs local observations from sensing with the physical constraints of contact. We propose a set of factors that ensure local consistency with tactile observations as well as enforcing physical plausibility, namely, that the estimated pose and contacts must respect the kinematic and force constraints of quasi-static rigid body interactions. We formalize our problem as a factor graph, allowing for efficient estimation. In our experiments, we demonstrate that our method outperforms existing geometric and contact-informed estimation pipelines, especially when only tactile information is available. Video results can be found at https://tacgraph.github.io/.

Related papers

• Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Hands [99.00248904465089]
  Dexterous hands differ in finger structure, contact surfaces, and sensor layouts.<n>We present Bench2Dex, a simulation benchmark for visuo-tactile bimanual manipulation across 12 dexterous hands.
  arXiv  Detail & Related papers  (2026-09-14T15:22:59Z)
• AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion [60.79507611737292]
  Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback.<n>Existing robotic grasping approaches rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact.<n>We propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement.
  arXiv  Detail & Related papers  (2026-08-06T16:29:11Z)
• BayesContact: Uncertain Pose Estimation via Visuo-Tactile Proposals and Simulation-based Inference [16.840783588655523]
  BayesContact is a framework for visuo-tactile pose estimation in peg-in-hole insertion.<n>It maintains a particle belief over object pose and fuses depth observations with force/torque-derived contact evidence.<n>It improves pose observability and insertion success over vision-only inference by 30%.
  arXiv  Detail & Related papers  (2026-07-17T17:00:42Z)
• Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation [74.87716678195099]
  Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation.<n>Despite their advantages, extracting fine-grained contact states necessary for contact-rich manipulation remains an open challenge.<n>We propose a motion-aware tactile representation to facilitate contact-rich manipulation.
  arXiv  Detail & Related papers  (2026-06-29T08:20:18Z)
• Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation [82.63833405368159]
  Generalizing tool manipulation requires both semantic planning and precise physical control.<n>We propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates.<n> Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization.
  arXiv  Detail & Related papers  (2026-02-14T16:05:08Z)
• Diffusion-based Inverse Model of a Distributed Tactile Sensor for Object Pose Estimation [45.95486121114164]
  We learn an inverse tactile sensor model using denoising diffusion to efficiently leverage tactile data for pose estimation.<n>The model is conditioned on tactile observations from a distributed tactile sensor and trained in simulation using a geometric sensor model.<n>For online pose estimation, we integrate the inverse model with a particle filter through a proposal scheme.
  arXiv  Detail & Related papers  (2026-01-19T17:36:28Z)
• Graph Neural Network Surrogates for Contacting Deformable Bodies with Necessary and Sufficient Contact Detection [0.0]
  Surrogate models for the rapid inference of nonlinear boundary value problems in mechanics are helpful in a broad range of engineering applications.<n>Existing methods are confined to rigid body contact or, at best, contact between rigid and soft objects with well-defined contact planes.<n>We present a graph neural network architecture that utilizes continuous collision detection and, for the first time, incorporates sufficient conditions designed for contact between soft deformable bodies.
  arXiv  Detail & Related papers  (2025-07-17T18:09:19Z)
• ViTaSCOPE: Visuo-tactile Implicit Representation for In-hand Pose and Extrinsic Contact Estimation [2.140861702387444]
  dexterous contact-rich object manipulation demands both object poses and external contact locations.<n>We present ViTaSCOPE: VisuoTac Simultaneous Contact and Object Estimation.
  arXiv  Detail & Related papers  (2025-06-13T21:35:58Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering [19.69677059281393]
  We present an approach to tactile pose estimation from the first touch for known objects. We create an object-agnostic map from real tactile observations to contact shapes. For a new object with known geometry, we learn a tailored perception model completely in simulation.
  arXiv  Detail & Related papers  (2020-12-09T18:00:35Z)
• Learning Tactile Models for Factor Graph-based Estimation [24.958055047646628]
  Vision-based tactile sensors provide rich, local image measurements at the point of contact. A single measurement contains limited information and multiple measurements are needed to infer latent object state. We propose a two-stage approach: first we learn local tactile observation models supervised with ground truth data, and then integrate these models along with physics and geometric factors within a factor graph.
  arXiv  Detail & Related papers  (2020-12-07T15:09:31Z)
• The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes? [57.366931129764815]
  We collect more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger.<n>Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.
  arXiv  Detail & Related papers  (2017-10-16T05:32:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.