Monocular Depth Estimation for Soft Visuotactile Sensors

• URL: http://arxiv.org/abs/2101.01677v1
• Date: Tue, 5 Jan 2021 17:51:11 GMT
• Title: Monocular Depth Estimation for Soft Visuotactile Sensors
• Authors: Rares Ambrus, Vitor Guizilini, Naveen Kuppuswamy, Andrew Beaulieu, Adrien Gaidon, Alex Alspach
• Abstract summary: We investigate the application of state-of-the-art monocular depth estimation to infer dense internal (tactile) depth maps directly from an internal single small IR imaging sensor. We show that deep networks typically used for long-range depth estimation (1-100m) can be effectively trained for precise predictions at a much shorter range (1-100mm) inside a mostly textureless deformable fluid-filled sensor. We propose a simple supervised learning process to train an object-agnostic network requiring less than 10 random poses in contact for less than 10 seconds for a small set of diverse objects.
• Score: 24.319343057803973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fluid-filled soft visuotactile sensors such as the Soft-bubbles alleviate key challenges for robust manipulation, as they enable reliable grasps along with the ability to obtain high-resolution sensory feedback on contact geometry and forces. Although they are simple in construction, their utility has been limited due to size constraints introduced by enclosed custom IR/depth imaging sensors to directly measure surface deformations. Towards mitigating this limitation, we investigate the application of state-of-the-art monocular depth estimation to infer dense internal (tactile) depth maps directly from the internal single small IR imaging sensor. Through real-world experiments, we show that deep networks typically used for long-range depth estimation (1-100m) can be effectively trained for precise predictions at a much shorter range (1-100mm) inside a mostly textureless deformable fluid-filled sensor. We propose a simple supervised learning process to train an object-agnostic network requiring less than 10 random poses in contact for less than 10 seconds for a small set of diverse objects (mug, wine glass, box, and fingers in our experiments). We show that our approach is sample-efficient, accurate, and generalizes across different objects and sensor configurations unseen at training time. Finally, we discuss the implications of our approach for the design of soft visuotactile sensors and grippers.

Related papers

• FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors [18.88211706267447]
  We tackle the problem of estimating 3D contact forces using vision-based tactile sensors. Our goal is to estimate contact forces over a large range (up to 15 N) on any objects while generalizing across different vision-based tactile sensors.
  arXiv  Detail & Related papers  (2024-10-02T21:28:19Z)
• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor [58.305341034419136]
  We present the first dense SLAM system with a monocular camera and a light-weight ToF sensor. We propose a multi-modal implicit scene representation that supports rendering both the signals from the RGB camera and light-weight ToF sensor. Experiments demonstrate that our system well exploits the signals of light-weight ToF sensors and achieves competitive results.
  arXiv  Detail & Related papers  (2023-08-28T07:56:13Z)
• UltraGlove: Hand Pose Estimation with Mems-Ultrasonic Sensors [14.257535961674021]
  We propose a novel and low-cost hand-tracking glove that utilizes several MEMS-ultrasonic sensors attached to the fingers. Our experimental results demonstrate that this approach is both accurate, size-agnostic, and robust to external interference.
  arXiv  Detail & Related papers  (2023-06-22T03:41:47Z)
• On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks [61.74608497496841]
  Training on inaccurate or corrupt data induces model bias and hampers generalisation capabilities. This paper investigates the effect of sensor errors for the dense 3D vision tasks of depth estimation and reconstruction.
  arXiv  Detail & Related papers  (2023-03-26T22:32:44Z)
• Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensors [4.886250215151643]
  We reason on the possible spatial configurations of the sensors along the object surface. We use selected sensors configurations to optimize over the space of 6D poses. We rank the obtained poses by penalizing those that are in collision with the sensors.
  arXiv  Detail & Related papers  (2023-01-31T14:35:26Z)
• The secret role of undesired physical effects in accurate shape sensing with eccentric FBGs [1.0805335573008565]
  Eccentric fiber Bragg gratings (FBG) are cheap and easy-to-fabricate shape sensors that are often interrogated with simple setups. Here, we present a novel technique to overcome these limitations and provide accurate and precise shape estimation.
  arXiv  Detail & Related papers  (2022-10-28T09:07:08Z)
• Learning Online Multi-Sensor Depth Fusion [100.84519175539378]
  SenFuNet is a depth fusion approach that learns sensor-specific noise and outlier statistics. We conduct experiments with various sensor combinations on the real-world CoRBS and Scene3D datasets.
  arXiv  Detail & Related papers  (2022-04-07T10:45:32Z)
• A soft thumb-sized vision-based sensor with accurate all-round force perception [19.905154050561013]
  Vision-based haptic sensors have emerged as a promising approach to robotic touch due to affordable high-resolution cameras and successful computer-vision techniques. We present a robust, soft, low-cost, vision-based, thumb-sized 3D haptic sensor named Insight.
  arXiv  Detail & Related papers  (2021-11-10T20:46:23Z)
• Event Guided Depth Sensing [50.997474285910734]
  We present an efficient bio-inspired event-camera-driven depth estimation algorithm. In our approach, we illuminate areas of interest densely, depending on the scene activity detected by the event camera. We show the feasibility of our approach in a simulated autonomous driving sequences and real indoor environments.
  arXiv  Detail & Related papers  (2021-10-20T11:41:11Z)
• Calibrating Self-supervised Monocular Depth Estimation [77.77696851397539]
  In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal. We show that incorporating prior information about the camera configuration and the environment, we can remove the scale ambiguity and predict depth directly, still using the self-supervised formulation and not relying on any additional sensors.
  arXiv  Detail & Related papers  (2020-09-16T14:35:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.