The RoboDepth Challenge: Methods and Advancements Towards Robust Depth
Estimation
- URL: http://arxiv.org/abs/2307.15061v1
- Date: Thu, 27 Jul 2023 17:59:56 GMT
- Title: The RoboDepth Challenge: Methods and Advancements Towards Robust Depth
Estimation
- Authors: Lingdong Kong and Yaru Niu and Shaoyuan Xie and Hanjiang Hu and Lai
Xing Ng and Benoit R. Cottereau and Ding Zhao and Liangjun Zhang and Hesheng
Wang and Wei Tsang Ooi and Ruijie Zhu and Ziyang Song and Li Liu and Tianzhu
Zhang and Jun Yu and Mohan Jing and Pengwei Li and Xiaohua Qi and Cheng Jin
and Yingfeng Chen and Jie Hou and Jie Zhang and Zhen Kan and Qiang Ling and
Liang Peng and Minglei Li and Di Xu and Changpeng Yang and Yuanqi Yao and
Gang Wu and Jian Kuai and Xianming Liu and Junjun Jiang and Jiamian Huang and
Baojun Li and Jiale Chen and Shuang Zhang and Sun Ao and Zhenyu Li and Runze
Chen and Haiyong Luo and Fang Zhao and Jingze Yu
- Abstract summary: We summarize the winning solutions from the RoboDepth Challenge.
The challenge was designed to facilitate and advance robust OoD depth estimation.
We hope this challenge could lay a solid foundation for future research on robust and reliable depth estimation.
- Score: 91.60650535480613
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Accurate depth estimation under out-of-distribution (OoD) scenarios, such as
adverse weather conditions, sensor failure, and noise contamination, is
desirable for safety-critical applications. Existing depth estimation systems,
however, suffer inevitably from real-world corruptions and perturbations and
are struggled to provide reliable depth predictions under such cases. In this
paper, we summarize the winning solutions from the RoboDepth Challenge -- an
academic competition designed to facilitate and advance robust OoD depth
estimation. This challenge was developed based on the newly established KITTI-C
and NYUDepth2-C benchmarks. We hosted two stand-alone tracks, with an emphasis
on robust self-supervised and robust fully-supervised depth estimation,
respectively. Out of more than two hundred participants, nine unique and
top-performing solutions have appeared, with novel designs ranging from the
following aspects: spatial- and frequency-domain augmentations, masked image
modeling, image restoration and super-resolution, adversarial training,
diffusion-based noise suppression, vision-language pre-training, learned model
ensembling, and hierarchical feature enhancement. Extensive experimental
analyses along with insightful observations are drawn to better understand the
rationale behind each design. We hope this challenge could lay a solid
foundation for future research on robust and reliable depth estimation and
beyond. The datasets, competition toolkit, workshop recordings, and source code
from the winning teams are publicly available on the challenge website.
Related papers
- Uncertainty-guided Optimal Transport in Depth Supervised Sparse-View 3D Gaussian [49.21866794516328]
3D Gaussian splatting has demonstrated impressive performance in real-time novel view synthesis.
Previous approaches have incorporated depth supervision into the training of 3D Gaussians to mitigate overfitting.
We introduce a novel method to supervise the depth distribution of 3D Gaussians, utilizing depth priors with integrated uncertainty estimates.
arXiv Detail & Related papers (2024-05-30T03:18:30Z) - STBA: Towards Evaluating the Robustness of DNNs for Query-Limited Black-box Scenario [50.37501379058119]
We propose the Spatial Transform Black-box Attack (STBA) to craft formidable adversarial examples in the query-limited scenario.
We show that STBA could effectively improve the imperceptibility of the adversarial examples and remarkably boost the attack success rate under query-limited settings.
arXiv Detail & Related papers (2024-03-30T13:28:53Z) - Stealing Stable Diffusion Prior for Robust Monocular Depth Estimation [33.140210057065644]
This paper introduces a novel approach named Stealing Stable Diffusion (SSD) prior for robust monocular depth estimation.
The approach addresses this limitation by utilizing stable diffusion to generate synthetic images that mimic challenging conditions.
The effectiveness of the approach is evaluated on nuScenes and Oxford RobotCar, two challenging public datasets.
arXiv Detail & Related papers (2024-03-08T05:06:31Z) - Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging
Scenarios [103.72094710263656]
This paper presents a novel approach that identifies and integrates dominant cross-modality depth features with a learning-based framework.
We propose a novel confidence loss steering a confidence predictor network to yield a confidence map specifying latent potential depth areas.
With the resulting confidence map, we propose a multi-modal fusion network that fuses the final depth in an end-to-end manner.
arXiv Detail & Related papers (2024-02-19T04:39:16Z) - RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions [7.359657743276515]
We introduce a comprehensive robustness test suite, RoboDepth, spanning 18 corruptions spanning three categories.
We benchmark 42 depth estimation models across indoor and outdoor scenes to assess their resilience to these corruptions.
Our findings underscore that, in the absence of a dedicated robustness evaluation framework, many leading depth estimation models may be susceptible to typical corruptions.
arXiv Detail & Related papers (2023-10-23T17:59:59Z) - EC-Depth: Exploring the consistency of self-supervised monocular depth estimation in challenging scenes [36.44321460703116]
EC-Depth is a novel self-supervised two-stage framework to achieve a robust depth estimation.
In the first stage, we propose depth consistency regularization to propagate reliable supervision from standard to challenging scenes.
In the second stage, we adopt a novel consistency-based pseudo-label filtering strategy to improve the quality of pseudo-labels.
arXiv Detail & Related papers (2023-10-12T05:34:45Z) - Unsupervised Scale-consistent Depth Learning from Video [131.3074342883371]
We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training.
Thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system.
The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training.
arXiv Detail & Related papers (2021-05-25T02:17:56Z) - Unsupervised Deep Persistent Monocular Visual Odometry and Depth
Estimation in Extreme Environments [7.197188771058501]
unsupervised deep learning approaches have received significant attention to estimate the depth and visual odometry (VO) from unlabelled monocular image sequences.
We propose an unsupervised monocular deep VO framework that predicts six-degrees-of-freedom pose camera motion and depth map of the scene from unlabelled RGB image sequences.
The proposed approach outperforms both traditional and state-of-the-art unsupervised deep VO methods providing better results for both pose estimation and depth recovery.
arXiv Detail & Related papers (2020-10-31T19:10:27Z) - On the uncertainty of self-supervised monocular depth estimation [52.13311094743952]
Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all.
We explore for the first time how to estimate the uncertainty for this task and how this affects depth accuracy.
We propose a novel peculiar technique specifically designed for self-supervised approaches.
arXiv Detail & Related papers (2020-05-13T09:00:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.