Tac2Pose: Tactile Object Pose Estimation from the First Touch

• URL: http://arxiv.org/abs/2204.11701v3
• Date: Thu, 14 Sep 2023 22:52:50 GMT
• Title: Tac2Pose: Tactile Object Pose Estimation from the First Touch
• Authors: Maria Bauza, Antonia Bronars, Alberto Rodriguez
• Abstract summary: We present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. We simulate the contact shapes that a dense set of object poses would produce on the sensor. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB tactile observations to binary contact shapes.
• Score: 6.321662423735226
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: In this paper, we present Tac2Pose, an object-specific approach to tactile pose estimation from the first touch for known objects. Given the object geometry, we learn a tailored perception model in simulation that estimates a probability distribution over possible object poses given a tactile observation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor, we match it against the pre-computed set using an object-specific embedding learned using contrastive learning. We obtain contact shapes from the sensor with an object-agnostic calibration step that maps RGB tactile observations to binary contact shapes. This mapping, which can be reused across object and sensor instances, is the only step trained with real sensor data. This results in a perception model that localizes objects from the first real tactile observation. Importantly, it produces pose distributions and can incorporate additional pose constraints coming from other perception systems, contacts, or priors. We provide quantitative results for 20 objects. Tac2Pose provides high accuracy pose estimations from distinctive tactile observations while regressing meaningful pose distributions to account for those contact shapes that could result from different object poses. We also test Tac2Pose on object models reconstructed from a 3D scanner, to evaluate the robustness to uncertainty in the object model. Finally, we demonstrate the advantages of Tac2Pose compared with three baseline methods for tactile pose estimation: directly regressing the object pose with a neural network, matching an observed contact to a set of possible contacts using a standard classification neural network, and direct pixel comparison of an observed contact with a set of possible contacts. Website: http://mcube.mit.edu/research/tac2pose.html

Related papers

• 3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects [13.58353565350936]
  We contribute methodology to jointly estimate the geometry and pose of objects grasped by a robot. Our method transforms the estimated geometry into the robot's coordinate frame. We empirically evaluate our approach on a robot manipulator holding a diverse set of real-world objects.
  arXiv  Detail & Related papers  (2024-07-14T21:02:55Z)
• Learning Explicit Contact for Implicit Reconstruction of Hand-held Objects from Monocular Images [59.49985837246644]
  We show how to model contacts in an explicit way to benefit the implicit reconstruction of hand-held objects. In the first part, we propose a new subtask of directly estimating 3D hand-object contacts from a single image. In the second part, we introduce a novel method to diffuse estimated contact states from the hand mesh surface to nearby 3D space.
  arXiv  Detail & Related papers  (2023-05-31T17:59:26Z)
• CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Network [66.24726878647543]
  Estimating the 6-DoF pose of a rigid object from a single RGB image is a crucial yet challenging task. Recent studies have shown the great potential of dense correspondence-based solutions. We propose a novel pose estimation algorithm named CheckerPose, which improves on three main aspects.
  arXiv  Detail & Related papers  (2023-03-29T17:30:53Z)
• FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedback [5.871946269300959]
  FingerSLAM is a closed-loop factor graph-based pose estimator that combines local tactile sensing at finger-tip and global vision sensing from a wrist-mount camera. We demonstrate reliable visuo-tactile pose estimation and shape reconstruction through quantitative and qualitative real-world evaluations on 6 objects that are unseen during training.
  arXiv  Detail & Related papers  (2023-03-14T15:48:47Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Detecting Human-Object Contact in Images [75.35017308643471]
  Humans constantly contact objects to move and perform tasks. There exists no robust method to detect contact between the body and the scene from an image. We build a new dataset of human-object contacts for images.
  arXiv  Detail & Related papers  (2023-03-06T18:56:26Z)
• Neural Correspondence Field for Object Pose Estimation [67.96767010122633]
  We propose a method for estimating the 6DoF pose of a rigid object with an available 3D model from a single RGB image. Unlike classical correspondence-based methods which predict 3D object coordinates at pixels of the input image, the proposed method predicts 3D object coordinates at 3D query points sampled in the camera frustum.
  arXiv  Detail & Related papers  (2022-07-30T01:48:23Z)
• Physically Plausible Pose Refinement using Fully Differentiable Forces [68.8204255655161]
  We propose an end-to-end differentiable model that refines pose estimates by learning the forces experienced by the object. By matching the learned net force to an estimate of net force based on finite differences of position, this model is able to find forces that accurately describe the movement of the object. We show this model successfully corrects poses and finds contact maps that better match the ground truth, despite not using any RGB or depth image data.
  arXiv  Detail & Related papers  (2021-05-17T23:33:04Z)
• Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering [19.69677059281393]
  We present an approach to tactile pose estimation from the first touch for known objects. We create an object-agnostic map from real tactile observations to contact shapes. For a new object with known geometry, we learn a tailored perception model completely in simulation.
  arXiv  Detail & Related papers  (2020-12-09T18:00:35Z)
• Learning Tactile Models for Factor Graph-based Estimation [24.958055047646628]
  Vision-based tactile sensors provide rich, local image measurements at the point of contact. A single measurement contains limited information and multiple measurements are needed to infer latent object state. We propose a two-stage approach: first we learn local tactile observation models supervised with ground truth data, and then integrate these models along with physics and geometric factors within a factor graph.
  arXiv  Detail & Related papers  (2020-12-07T15:09:31Z)
• Contact Area Detector using Cross View Projection Consistency for COVID-19 Projects [7.539495357219132]
  We show that the contact between an object and a static surface can be identified by projecting the object onto the static surface through two different viewpoints. This simple method can be easily adapted to real-life applications.
  arXiv  Detail & Related papers  (2020-08-18T02:57:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.