GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger

• URL: http://arxiv.org/abs/2106.08851v1
• Date: Wed, 16 Jun 2021 15:15:29 GMT
• Title: GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger
• Authors: Shaoxiong Wang, Yu She, Branden Romero, Edward Adelson
• Abstract summary: GelSight Wedge sensor is optimized to have a compact shape for robot fingers, while achieving high-resolution 3D reconstruction. We show the effectiveness and potential of the reconstructed 3D geometry for pose tracking in the 3D space.
• Score: 8.047951969722794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-based tactile sensors have the potential to provide important contact geometry to localize the objective with visual occlusion. However, it is challenging to measure high-resolution 3D contact geometry for a compact robot finger, to simultaneously meet optical and mechanical constraints. In this work, we present the GelSight Wedge sensor, which is optimized to have a compact shape for robot fingers, while achieving high-resolution 3D reconstruction. We evaluate the 3D reconstruction under different lighting configurations, and extend the method from 3 lights to 1 or 2 lights. We demonstrate the flexibility of the design by shrinking the sensor to the size of a human finger for fine manipulation tasks. We also show the effectiveness and potential of the reconstructed 3D geometry for pose tracking in the 3D space.

Related papers

• Incorporating dense metric depth into neural 3D representations for view synthesis and relighting [25.028859317188395]
  In robotic applications, dense metric depth can often be measured directly using stereo and illumination can be controlled. In this work we demonstrate a method to incorporate dense metric depth into the training of neural 3D representations. We also discuss a multi-flash stereo camera system developed to capture the necessary data for our pipeline and show results on relighting and view synthesis.
  arXiv  Detail & Related papers  (2024-09-04T20:21:13Z)
• Towards 3D Vision with Low-Cost Single-Photon Cameras [24.711165102559438]
  We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. Our work draws a connection between image-based modeling and active range scanning and is a step towards 3D vision with single-photon cameras.
  arXiv  Detail & Related papers  (2024-03-26T15:40:05Z)
• TouchSDF: A DeepSDF Approach for 3D Shape Reconstruction using Vision-Based Tactile Sensing [29.691786688595762]
  Humans rely on their visual and tactile senses to develop a comprehensive 3D understanding of their physical environment. We propose TouchSDF, a Deep Learning approach for tactile 3D shape reconstruction. Our technique consists of two components: (1) a Convolutional Neural Network that maps tactile images into local meshes representing the surface at the touch location, and (2) an implicit neural function that predicts a signed distance function to extract the desired 3D shape.
  arXiv  Detail & Related papers  (2023-11-21T13:43:06Z)
• MagicDrive: Street View Generation with Diverse 3D Geometry Control [82.69871576797166]
  We introduce MagicDrive, a novel street view generation framework, offering diverse 3D geometry controls. Our design incorporates a cross-view attention module, ensuring consistency across multiple camera views.
  arXiv  Detail & Related papers  (2023-10-04T06:14:06Z)
• gSDF: Geometry-Driven Signed Distance Functions for 3D Hand-Object Reconstruction [94.46581592405066]
  We exploit the hand structure and use it as guidance for SDF-based shape reconstruction. We predict kinematic chains of pose transformations and align SDFs with highly-articulated hand poses.
  arXiv  Detail & Related papers  (2023-04-24T10:05:48Z)
• OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis [81.70922087960271]
  We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images. Our model can synthesize more preferable identity-preserved 3D heads with compelling dynamic details compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-27T18:36:53Z)
• DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction [0.0]
  Vision-based tactile sensors have been widely used as rich tactile feedback has been correlated with increased performance in manipulation tasks. Existing tactile sensor solutions with high resolution have limitations that include low accuracy, expensive components, or lack of scalability. This paper proposes an inexpensive, scalable, and compact tactile sensor with high-resolution surface deformation modeling for surface reconstruction of the 3D sensor surface.
  arXiv  Detail & Related papers  (2022-01-04T22:26:14Z)
• EGFN: Efficient Geometry Feature Network for Fast Stereo 3D Object Detection [51.52496693690059]
  Fast stereo based 3D object detectors lag far behind high-precision oriented methods in accuracy. We argue that the main reason is the missing or poor 3D geometry feature representation in fast stereo based methods. The proposed EGFN outperforms YOLOStsereo3D, the advanced fast method, by 5.16% on mAP$_3d$ at the cost of merely additional 12 ms.
  arXiv  Detail & Related papers  (2021-11-28T05:25:36Z)
• Active 3D Shape Reconstruction from Vision and Touch [66.08432412497443]
  Humans build 3D understandings of the world through active object exploration, using jointly their senses of vision and touch. In 3D shape reconstruction, most recent progress has relied on static datasets of limited sensory data such as RGB images, depth maps or haptic readings. We introduce a system composed of: 1) a haptic simulator leveraging high spatial resolution vision-based tactile sensors for active touching of 3D objects; 2) a mesh-based 3D shape reconstruction model that relies on tactile or visuotactile priors to guide the shape exploration; and 3) a set of data-driven solutions with either tactile or visuo
  arXiv  Detail & Related papers  (2021-07-20T15:56:52Z)
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection [103.5060007382646]
  We propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization. We design an end-to-end learnable architecture that exploits continuous convolutions to fuse image and LIDAR feature maps at different levels of resolution.
  arXiv  Detail & Related papers  (2020-12-20T18:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.