Image Masking for Robust Self-Supervised Monocular Depth Estimation
- URL: http://arxiv.org/abs/2210.02357v1
- Date: Wed, 5 Oct 2022 15:57:53 GMT
- Title: Image Masking for Robust Self-Supervised Monocular Depth Estimation
- Authors: Hemang Chawla, Kishaan Jeeveswaran, Elahe Arani, Bahram Zonooz
- Abstract summary: Self-supervised monocular depth estimation is a salient task for 3D scene understanding.
We propose MIMDepth, a method that adapts masked image modeling for self-supervised monocular depth estimation.
- Score: 12.435468563991174
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Self-supervised monocular depth estimation is a salient task for 3D scene
understanding. Learned jointly with monocular ego-motion estimation, several
methods have been proposed to predict accurate pixel-wise depth without using
labeled data. Nevertheless, these methods focus on improving performance under
ideal conditions without natural or digital corruptions. A general absence of
occlusions is assumed even for object-specific depth estimation. These methods
are also vulnerable to adversarial attacks, which is a pertinent concern for
their reliable deployment on robots and autonomous driving systems. We propose
MIMDepth, a method that adapts masked image modeling (MIM) for self-supervised
monocular depth estimation. While MIM has been used to learn generalizable
features during pre-training, we show how it could be adapted for direct
training of monocular depth estimation. Our experiments show that MIMDepth is
more robust to noise, blur, weather conditions, digital artifacts, occlusions,
as well as untargeted and targeted adversarial attacks.
Related papers
- Fully Self-Supervised Depth Estimation from Defocus Clue [79.63579768496159]
We propose a self-supervised framework that estimates depth purely from a sparse focal stack.
We show that our framework circumvents the needs for the depth and AIF image ground-truth, and receives superior predictions.
arXiv Detail & Related papers (2023-03-19T19:59:48Z) - SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for
Dynamic Scenes [58.89295356901823]
Self-supervised monocular depth estimation has shown impressive results in static scenes.
It relies on the multi-view consistency assumption for training networks, however, that is violated in dynamic object regions.
We introduce an external pretrained monocular depth estimation model for generating single-image depth prior.
Our model can predict sharp and accurate depth maps, even when training from monocular videos of highly-dynamic scenes.
arXiv Detail & Related papers (2022-11-07T16:17:47Z) - SelfTune: Metrically Scaled Monocular Depth Estimation through
Self-Supervised Learning [53.78813049373321]
We propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation.
Our approach is useful for various applications such as mobile robot navigation and is applicable to diverse environments.
arXiv Detail & Related papers (2022-03-10T12:28:42Z) - Variational Monocular Depth Estimation for Reliability Prediction [12.951621755732544]
Self-supervised learning for monocular depth estimation is widely investigated as an alternative to supervised learning approach.
Previous works have successfully improved the accuracy of depth estimation by modifying the model structure.
In this paper, we theoretically formulate a variational model for the monocular depth estimation to predict the reliability of the estimated depth image.
arXiv Detail & Related papers (2020-11-24T06:23:51Z) - Adversarial Patch Attacks on Monocular Depth Estimation Networks [7.089737454146505]
We propose a method of adversarial patch attack on monocular depth estimation.
We generate artificial patterns that can fool the target methods into estimating an incorrect depth for the regions where the patterns are placed.
Our method can be implemented in the real world by physically placing the printed patterns in real scenes.
arXiv Detail & Related papers (2020-10-06T22:56:22Z) - Adaptive confidence thresholding for monocular depth estimation [83.06265443599521]
We propose a new approach to leverage pseudo ground truth depth maps of stereo images generated from self-supervised stereo matching methods.
The confidence map of the pseudo ground truth depth map is estimated to mitigate performance degeneration by inaccurate pseudo depth maps.
Experimental results demonstrate superior performance to state-of-the-art monocular depth estimation methods.
arXiv Detail & Related papers (2020-09-27T13:26:16Z) - Calibrating Self-supervised Monocular Depth Estimation [77.77696851397539]
In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal.
We show that incorporating prior information about the camera configuration and the environment, we can remove the scale ambiguity and predict depth directly, still using the self-supervised formulation and not relying on any additional sensors.
arXiv Detail & Related papers (2020-09-16T14:35:45Z) - Self-Supervised Learning for Monocular Depth Estimation from Aerial
Imagery [0.20072624123275526]
We present a method for self-supervised learning for monocular depth estimation from aerial imagery.
For this, we only use an image sequence from a single moving camera and learn to simultaneously estimate depth and pose information.
By sharing the weights between pose and depth estimation, we achieve a relatively small model, which favors real-time application.
arXiv Detail & Related papers (2020-08-17T12:20:46Z) - Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object
Problem by Semantic Guidance [36.73303869405764]
Self-supervised monocular depth estimation presents a powerful method to obtain 3D scene information from single camera images.
We present a new self-supervised semantically-guided depth estimation (SGDepth) method to deal with moving dynamic-class (DC) objects.
arXiv Detail & Related papers (2020-07-14T09:47:27Z) - On the uncertainty of self-supervised monocular depth estimation [52.13311094743952]
Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all.
We explore for the first time how to estimate the uncertainty for this task and how this affects depth accuracy.
We propose a novel peculiar technique specifically designed for self-supervised approaches.
arXiv Detail & Related papers (2020-05-13T09:00:55Z) - Self-supervised Monocular Trained Depth Estimation using Self-attention
and Discrete Disparity Volume [19.785343302320918]
We propose two new ideas to improve self-supervised monocular trained depth estimation: 1) self-attention, and 2) discrete disparity prediction.
We show that the extension of the state-of-the-art self-supervised monocular trained depth estimator Monodepth2 with these two ideas allows us to design a model that produces the best results in the field in KITTI 2015 and Make3D.
arXiv Detail & Related papers (2020-03-31T04:48:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.