TransTouch: Learning Transparent Objects Depth Sensing Through Sparse
Touches
- URL: http://arxiv.org/abs/2309.09427v1
- Date: Mon, 18 Sep 2023 01:55:17 GMT
- Title: TransTouch: Learning Transparent Objects Depth Sensing Through Sparse
Touches
- Authors: Liuyu Bian, Pengyang Shi, Weihang Chen, Jing Xu, Li Yi, Rui Chen
- Abstract summary: We propose a method to finetune a stereo network with sparse depth labels automatically collected using a probing system with tactile feedback.
We show that our method can significantly improve real-world depth sensing accuracy, especially for transparent objects.
- Score: 23.87056600709768
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Transparent objects are common in daily life. However, depth sensing for
transparent objects remains a challenging problem. While learning-based methods
can leverage shape priors to improve the sensing quality, the labor-intensive
data collection in the real world and the sim-to-real domain gap restrict these
methods' scalability. In this paper, we propose a method to finetune a stereo
network with sparse depth labels automatically collected using a probing system
with tactile feedback. We present a novel utility function to evaluate the
benefit of touches. By approximating and optimizing the utility function, we
can optimize the probing locations given a fixed touching budget to better
improve the network's performance on real objects. We further combine tactile
depth supervision with a confidence-based regularization to prevent
over-fitting during finetuning. To evaluate the effectiveness of our method, we
construct a real-world dataset including both diffuse and transparent objects.
Experimental results on this dataset show that our method can significantly
improve real-world depth sensing accuracy, especially for transparent objects.
Related papers
- RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface
Normal Estimation and Manipulation [50.10282876199739]
This paper introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects.
It integrates the RFNet, which predicts refractive flow, object mask, and boundaries, followed by the F2Net, which estimates surface normal from the refractive flow.
A real-world robot grasping task witnesses an 83% success rate, proving that refractive flow can help enable direct sim-to-real transfer.
arXiv Detail & Related papers (2023-11-21T07:19:47Z) - Transparent Object Tracking with Enhanced Fusion Module [56.403878717170784]
We propose a new tracker architecture that uses our fusion techniques to achieve superior results for transparent object tracking.
Our results and the implementation of code will be made publicly available at https://github.com/kalyan05TOTEM.
arXiv Detail & Related papers (2023-09-13T03:52:09Z) - Invariant Slot Attention: Object Discovery with Slot-Centric Reference
Frames [18.84636947819183]
Slot-based neural networks that learn about objects in a self-supervised manner have made exciting progress.
We present a simple yet highly effective method for incorporating spatial symmetries via slot-centric reference frames.
We evaluate our method on a range of synthetic object discovery benchmarks namely CLEVR, Tetrominoes, CLEVR, Objects Room and MultiShapeNet.
arXiv Detail & Related papers (2023-02-09T23:25:28Z) - AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
We propose a novel 3D detection framework that associates intact features for objects via domain adaptation.
We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
arXiv Detail & Related papers (2022-08-24T16:54:38Z) - High-resolution Iterative Feedback Network for Camouflaged Object
Detection [128.893782016078]
Spotting camouflaged objects that are visually assimilated into the background is tricky for object detection algorithms.
We aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries.
We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner.
arXiv Detail & Related papers (2022-03-22T11:20:21Z) - TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth
Completion and Grasping [46.6058840385155]
We contribute a large-scale real-world dataset for transparent object depth completion.
Our dataset contains 57,715 RGB-D images from 130 different scenes.
We propose an end-to-end depth completion network, which takes the RGB image and the inaccurate depth map as inputs and outputs a refined depth map.
arXiv Detail & Related papers (2022-02-17T06:50:20Z) - Self-Guided Instance-Aware Network for Depth Completion and Enhancement [6.319531161477912]
Existing methods directly interpolate the missing depth measurements based on pixel-wise image content and the corresponding neighboring depth values.
We propose a novel self-guided instance-aware network (SG-IANet) that utilize self-guided mechanism to extract instance-level features that is needed for depth restoration.
arXiv Detail & Related papers (2021-05-25T19:41:38Z) - FakeMix Augmentation Improves Transparent Object Detection [24.540569928274984]
We propose a novel content-dependent data augmentation method termed FakeMix to overcome the boundary-related imbalance problem.
We also present AdaptiveASPP, an enhanced version of ASPP, that can capture multi-scale and cross-modality features dynamically.
arXiv Detail & Related papers (2021-03-24T15:51:37Z) - Accurate RGB-D Salient Object Detection via Collaborative Learning [101.82654054191443]
RGB-D saliency detection shows impressive ability on some challenge scenarios.
We propose a novel collaborative learning framework where edge, depth and saliency are leveraged in a more efficient way.
arXiv Detail & Related papers (2020-07-23T04:33:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.