The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?

• URL: http://arxiv.org/abs/1710.05512v2
• Date: Tue, 04 Mar 2025 22:29:30 GMT
• Title: The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
• Authors: Roberto Calandra, Andrew Owens, Manu Upadhyaya, Wenzhen Yuan, Justin Lin, Edward H. Adelson, Sergey Levine,
• Abstract summary: We collect more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger.<n>Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.
• Score: 57.366931129764815
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A successful grasp requires careful balancing of the contact forces. Deducing whether a particular grasp will be successful from indirect measurements, such as vision, is therefore quite challenging, and direct sensing of contacts through touch sensing provides an appealing avenue toward more successful and consistent robotic grasping. However, in order to fully evaluate the value of touch sensing for grasp outcome prediction, we must understand how touch sensing can influence outcome prediction accuracy when combined with other modalities. Doing so using conventional model-based techniques is exceptionally difficult. In this work, we investigate the question of whether touch sensing aids in predicting grasp outcomes within a multimodal sensing framework that combines vision and touch. To that end, we collected more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger, and evaluated visuo-tactile deep neural network models to directly predict grasp outcomes from either modality individually, and from both modalities together. Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.

Related papers

• AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors [11.506370451126378]
  Visuo-tactile sensors aim to emulate human tactile perception, enabling robots to understand and manipulate objects. We introduce TacQuad, an aligned multi-modal tactile multi-sensor dataset from four different visuo-tactile sensors. We propose AnyTouch, a unified static-dynamic multi-sensor representation learning framework with a multi-level structure.
  arXiv  Detail & Related papers  (2025-02-15T08:33:25Z)
• Learning Visuotactile Skills with Two Multifingered Hands [80.99370364907278]
  We explore learning from human demonstrations using a bimanual system with multifingered hands and visuotactile data. Our results mark a promising step forward in bimanual multifingered manipulation from visuotactile data.
  arXiv  Detail & Related papers  (2024-04-25T17:59:41Z)
• Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation [57.60490773016364]
  We combine vision and touch sensing on a multi-fingered hand to estimate an object's pose and shape during in-hand manipulation. Our method, NeuralFeels, encodes object geometry by learning a neural field online and jointly tracks it by optimizing a pose graph problem. Our results demonstrate that touch, at the very least, refines and, at the very best, disambiguates visual estimates during in-hand manipulation.
  arXiv  Detail & Related papers  (2023-12-20T22:36:37Z)
• A model-free approach to fingertip slip and disturbance detection for grasp stability inference [0.0]
  We propose a method for assessing grasp stability using tactile sensing. We use highly sensitive Uskin tactile sensors mounted on an Allegro hand to test and validate our method.
  arXiv  Detail & Related papers  (2023-11-22T09:04:26Z)
• Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor [14.492202828369127]
  We leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks.<n>We introduce two algorithmic contributions, tactile force matching and learned mode switching, as complimentary methods for improving IL.<n>Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%.
  arXiv  Detail & Related papers  (2023-11-02T14:02:42Z)
• Bayesian and Neural Inference on LSTM-based Object Recognition from Tactile and Kinesthetic Information [0.0]
  Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations) This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results.
  arXiv  Detail & Related papers  (2023-06-10T12:29:23Z)
• Tactile-Filter: Interactive Tactile Perception for Part Mating [54.46221808805662]
  Humans rely on touch and tactile sensing for a lot of dexterous manipulation tasks. vision-based tactile sensors are being widely used for various robotic perception and control tasks. We present a method for interactive perception using vision-based tactile sensors for a part mating task.
  arXiv  Detail & Related papers  (2023-03-10T16:27:37Z)
• Learning to Detect Slip with Barometric Tactile Sensors and a Temporal Convolutional Neural Network [7.346580429118843]
  We present a learning-based method to detect slip using barometric tactile sensors. We train a temporal convolution neural network to detect slip, achieving high detection accuracies. We argue that barometric tactile sensing technology, combined with data-driven learning, is suitable for many manipulation tasks such as slip compensation.
  arXiv  Detail & Related papers  (2022-02-19T08:21:56Z)
• Elastic Tactile Simulation Towards Tactile-Visual Perception [58.44106915440858]
  We propose Elastic Interaction of Particles (EIP) for tactile simulation. EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact. We further propose a tactile-visual perception network that enables information fusion between tactile data and visual images.
  arXiv  Detail & Related papers  (2021-08-11T03:49:59Z)
• Under Pressure: Learning to Detect Slip with Barometric Tactile Sensors [7.35805050004643]
  We present a learning-based method to detect slip using barometric tactile sensors. We are able to achieve slip detection accuracies of greater than 91%. We show that barometric tactile sensing technology, combined with data-driven learning, is potentially suitable for many complex manipulation tasks.
  arXiv  Detail & Related papers  (2021-03-24T19:29:03Z)
• OmniTact: A Multi-Directional High Resolution Touch Sensor [109.28703530853542]
  Existing tactile sensors are either flat, have small sensitive fields or only provide low-resolution signals. We introduce OmniTact, a multi-directional high-resolution tactile sensor. We evaluate the capabilities of OmniTact on a challenging robotic control task.
  arXiv  Detail & Related papers  (2020-03-16T01:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.