Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Driving

• URL: http://arxiv.org/abs/2607.21526v1
• Date: Thu, 23 Jul 2026 17:11:33 GMT
• Title: Boosting Robustness for All-Weather Self-Supervised Depth Estimation in Autonomous Driving
• Abstract summary: Adverse weather conditions can distort pixel correspondences and violate the assumptions embedded in the self-supervised loss function.<n>While radar is a widely adopted sensor in adverse weather conditions, the sparse distribution of radar points in the Point of View (POV) poses challenges for self-supervised fusion.<n>We propose a novel self-training pipeline using unpaired real all-weather data through multi-teacher distillation and robust radar fusion.
• Score: 46.17295445504376
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Self-supervised depth estimation is challenging for safe autonomous driving under various adverse weather conditions due to sensor perception degradation. These challenges arise from two main aspects. Firstly, adverse conditions can distort pixel correspondences and violate the assumptions embedded in the self-supervised loss function, leading to erroneous depth predictions. Secondly, while radar is a widely adopted sensor in adverse weather conditions, the sparse distribution of radar points in the Point of View (POV) poses challenges for self-supervised fusion. To address these issues, we introduce a novel self-training pipeline using unpaired real all-weather data through multi-teacher distillation and robust radar fusion. We propose the Uncertainty-Aware Multi-Teacher Distillation method to generate diverse teacher models with different adverse condition inputs, and then employ uncertainty modeling to weigh the knowledge distillation loss. Additionally, we design the POV-BEV Radar Fusion approach, which leverages camera-pixel ray constraints to establish connections between the camera's Point of View (POV) and the radar's Bird's-Eye View (BEV). This approach enables the utilization of denser radar points, effectively capturing the complementary perspectives of both POV and BEV. Extensive quantitative and qualitative experiments demonstrate the robustness of our proposed method on all-weather datasets, achieving state-of-the-art performance. Our code and models are available at https://github.com/MICLAB-BUPT/RobustDepth.

Related papers

• DinoRADE: Full Spectral Radar-Camera Fusion with Vision Foundation Model Features for Multi-class Object Detection in Adverse Weather [4.199844472131922]
  DinoRADE is a Radar-centered detection pipeline that processes dense Radar tensors and aggregates vision features around transformed reference points in the camera perspective.<n>We present a comprehensive performance evaluation on the K-Radar dataset in all weather conditions and are among the first to report detection performance individually for five object classes.
  arXiv  Detail & Related papers  (2026-04-09T10:46:41Z)
• SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather [33.428611256223256]
  We introduce a novel multi-sensor fusion approach tailored to adverse weather conditions.<n>In addition to fusing RGB and LiDAR sensors, our sensor fusion stack is also capable of learning from NIR camera and radar modalities.<n>Our method improves the reliability of multimodal sensor fusion in autonomous vehicles under challenging weather conditions.
  arXiv  Detail & Related papers  (2025-08-22T14:20:46Z)
• TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion [54.46664104437454]
  We propose TacoDepth, an efficient and accurate Radar-Camera depth estimation model with one-stage fusion.<n>Specifically, the graph-based Radar structure extractor and the pyramid-based Radar fusion module are designed.<n>Compared with the previous state-of-the-art approach, TacoDepth improves depth accuracy and processing speed by 12.8% and 91.8%.
  arXiv  Detail & Related papers  (2025-04-16T05:25:04Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance.<n>Radar suffers from noise and positional ambiguity.<n>We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• RIDERS: Radar-Infrared Depth Estimation for Robust Sensing [22.10378524682712]
  Adverse weather conditions pose significant challenges to accurate dense depth estimation. We present a novel approach for robust metric depth estimation by fusing a millimeter-wave Radar and a monocular infrared thermal camera. Our method achieves exceptional visual quality and accurate metric estimation by addressing the challenges of ambiguity and misalignment.
  arXiv  Detail & Related papers  (2024-02-03T07:14:43Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges [0.021665899581403605]
  Radar is a key component of the suite of perception sensors used for autonomous vehicles. It is characterized by low resolution, sparsity, clutter, high uncertainty, and lack of good datasets. Current radar models are often influenced by lidar and vision models, which are focused on optical features that are relatively weak in radar data.
  arXiv  Detail & Related papers  (2023-06-15T17:37:52Z)
• RaLiBEV: Radar and LiDAR BEV Fusion Learning for Anchor Box Free Object Detection Systems [13.046347364043594]
  In autonomous driving, LiDAR and radar are crucial for environmental perception. Recent state-of-the-art works reveal that the fusion of radar and LiDAR can lead to robust detection in adverse weather. We propose a bird's-eye view fusion learning-based anchor box-free object detection system.
  arXiv  Detail & Related papers  (2022-11-11T10:24:42Z)
• R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes [69.6715406227469]
  Self-supervised monocular depth estimation in driving scenarios has achieved comparable performance to supervised approaches. We present R4Dyn, a novel set of techniques to use cost-efficient radar data on top of a self-supervised depth estimation framework.
  arXiv  Detail & Related papers  (2021-08-10T17:57:03Z)
• Radar Camera Fusion via Representation Learning in Autonomous Driving [4.278336455989584]
  Key to successful radar-camera fusion is accurate data association. Traditional rule-based association methods are susceptible to performance degradation in challenging scenarios and failure in corner cases. We propose to address rad-cam association via deep representation learning, to explore feature-level interaction and global reasoning.
  arXiv  Detail & Related papers  (2021-03-14T01:32:03Z)
• Physically Realizable Adversarial Examples for LiDAR Object Detection [72.0017682322147]
  We present a method to generate universal 3D adversarial objects to fool LiDAR detectors. In particular, we demonstrate that placing an adversarial object on the rooftop of any target vehicle to hide the vehicle entirely from LiDAR detectors with a success rate of 80%. This is one step closer towards safer self-driving under unseen conditions from limited training data.
  arXiv  Detail & Related papers  (2020-04-01T16:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.