The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation

• URL: http://arxiv.org/abs/2307.15061v2
• Date: Tue, 24 Sep 2024 23:49:07 GMT
• Title: The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation
• Authors: Lingdong Kong, Yaru Niu, Shaoyuan Xie, Hanjiang Hu, Lai Xing Ng, Benoit R. Cottereau, Liangjun Zhang, Hesheng Wang, Wei Tsang Ooi, Ruijie Zhu, Ziyang Song, Li Liu, Tianzhu Zhang, Jun Yu, Mohan Jing, Pengwei Li, Xiaohua Qi, Cheng Jin, Yingfeng Chen, Jie Hou, Jie Zhang, Zhen Kan, Qiang Ling, Liang Peng, Minglei Li, Di Xu, Changpeng Yang, Yuanqi Yao, Gang Wu, Jian Kuai, Xianming Liu, Junjun Jiang, Jiamian Huang, Baojun Li, Jiale Chen, Shuang Zhang, Sun Ao, Zhenyu Li, Runze Chen, Haiyong Luo, Fang Zhao, Jingze Yu,
• Abstract summary: We summarize the winning solutions from the RoboDepth Challenge. The challenge was designed to facilitate and advance robust OoD depth estimation. We hope this challenge could lay a solid foundation for future research on robust and reliable depth estimation.
• Score: 97.63185634482552
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Accurate depth estimation under out-of-distribution (OoD) scenarios, such as adverse weather conditions, sensor failure, and noise contamination, is desirable for safety-critical applications. Existing depth estimation systems, however, suffer inevitably from real-world corruptions and perturbations and are struggled to provide reliable depth predictions under such cases. In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation. This challenge was developed based on the newly established KITTI-C and NYUDepth2-C benchmarks. We hosted two stand-alone tracks, with an emphasis on robust self-supervised and robust fully-supervised depth estimation, respectively. Out of more than two hundred participants, nine unique and top-performing solutions have appeared, with novel designs ranging from the following aspects: spatial- and frequency-domain augmentations, masked image modeling, image restoration and super-resolution, adversarial training, diffusion-based noise suppression, vision-language pre-training, learned model ensembling, and hierarchical feature enhancement. Extensive experimental analyses along with insightful observations are drawn to better understand the rationale behind each design. We hope this challenge could lay a solid foundation for future research on robust and reliable depth estimation and beyond. The datasets, competition toolkit, workshop recordings, and source code from the winning teams are publicly available on the challenge website.

Related papers

• Uncertainty-guided Optimal Transport in Depth Supervised Sparse-View 3D Gaussian [49.21866794516328]
  3D Gaussian splatting has demonstrated impressive performance in real-time novel view synthesis. Previous approaches have incorporated depth supervision into the training of 3D Gaussians to mitigate overfitting. We introduce a novel method to supervise the depth distribution of 3D Gaussians, utilizing depth priors with integrated uncertainty estimates.
  arXiv  Detail & Related papers  (2024-05-30T03:18:30Z)
• STBA: Towards Evaluating the Robustness of DNNs for Query-Limited Black-box Scenario [50.37501379058119]
  We propose the Spatial Transform Black-box Attack (STBA) to craft formidable adversarial examples in the query-limited scenario. We show that STBA could effectively improve the imperceptibility of the adversarial examples and remarkably boost the attack success rate under query-limited settings.
  arXiv  Detail & Related papers  (2024-03-30T13:28:53Z)
• Stealing Stable Diffusion Prior for Robust Monocular Depth Estimation [33.140210057065644]
  This paper introduces a novel approach named Stealing Stable Diffusion (SSD) prior for robust monocular depth estimation. The approach addresses this limitation by utilizing stable diffusion to generate synthetic images that mimic challenging conditions. The effectiveness of the approach is evaluated on nuScenes and Oxford RobotCar, two challenging public datasets.
  arXiv  Detail & Related papers  (2024-03-08T05:06:31Z)
• Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios [103.72094710263656]
  This paper presents a novel approach that identifies and integrates dominant cross-modality depth features with a learning-based framework. We propose a novel confidence loss steering a confidence predictor network to yield a confidence map specifying latent potential depth areas. With the resulting confidence map, we propose a multi-modal fusion network that fuses the final depth in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-02-19T04:39:16Z)
• RoboDepth: Robust Out-of-Distribution Depth Estimation under Corruptions [7.359657743276515]
  We introduce a comprehensive robustness test suite, RoboDepth, spanning 18 corruptions spanning three categories. We benchmark 42 depth estimation models across indoor and outdoor scenes to assess their resilience to these corruptions. Our findings underscore that, in the absence of a dedicated robustness evaluation framework, many leading depth estimation models may be susceptible to typical corruptions.
  arXiv  Detail & Related papers  (2023-10-23T17:59:59Z)
• EC-Depth: Exploring the consistency of self-supervised monocular depth estimation in challenging scenes [36.44321460703116]
  EC-Depth is a novel self-supervised two-stage framework to achieve a robust depth estimation. In the first stage, we propose depth consistency regularization to propagate reliable supervision from standard to challenging scenes. In the second stage, we adopt a novel consistency-based pseudo-label filtering strategy to improve the quality of pseudo-labels.
  arXiv  Detail & Related papers  (2023-10-12T05:34:45Z)
• Unsupervised Scale-consistent Depth Learning from Video [131.3074342883371]
  We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training. Thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system. The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training.
  arXiv  Detail & Related papers  (2021-05-25T02:17:56Z)
• Unsupervised Deep Persistent Monocular Visual Odometry and Depth Estimation in Extreme Environments [7.197188771058501]
  unsupervised deep learning approaches have received significant attention to estimate the depth and visual odometry (VO) from unlabelled monocular image sequences. We propose an unsupervised monocular deep VO framework that predicts six-degrees-of-freedom pose camera motion and depth map of the scene from unlabelled RGB image sequences. The proposed approach outperforms both traditional and state-of-the-art unsupervised deep VO methods providing better results for both pose estimation and depth recovery.
  arXiv  Detail & Related papers  (2020-10-31T19:10:27Z)
• On the uncertainty of self-supervised monocular depth estimation [52.13311094743952]
  Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all. We explore for the first time how to estimate the uncertainty for this task and how this affects depth accuracy. We propose a novel peculiar technique specifically designed for self-supervised approaches.
  arXiv  Detail & Related papers  (2020-05-13T09:00:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.