Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion

• URL: http://arxiv.org/abs/2312.00844v3
• Date: Thu, 18 Jul 2024 16:05:55 GMT
• Title: Sparse Beats Dense: Rethinking Supervision in Radar-Camera Depth Completion
• Authors: Huadong Li, Minhao Jing, Jiajun Liang, Haoqiang Fan, Renhe Ji,
• Abstract summary: We present a new method with sparse LiDAR supervision to outperform previous dense LiDAR supervision methods in both accuracy and speed. We find that depth completion models usually output depth maps containing significant stripe-like artifacts when trained by sparse LiDAR supervision. Our framework with sparse supervision outperforms the state-of-the-art dense supervision methods with 11.6% improvement in Mean Absolute Error (MAE) and 1.6x speedup in Frame Per Second (FPS).
• Score: 18.0877558432168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is widely believed that sparse supervision is worse than dense supervision in the field of depth completion, but the underlying reasons for this are rarely discussed. To this end, we revisit the task of radar-camera depth completion and present a new method with sparse LiDAR supervision to outperform previous dense LiDAR supervision methods in both accuracy and speed. Specifically, when trained by sparse LiDAR supervision, depth completion models usually output depth maps containing significant stripe-like artifacts. We find that such a phenomenon is caused by the implicitly learned positional distribution pattern from sparse LiDAR supervision, termed as LiDAR Distribution Leakage (LDL) in this paper. Based on such understanding, we present a novel Disruption-Compensation radar-camera depth completion framework to address this issue. The Disruption part aims to deliberately disrupt the learning of LiDAR distribution from sparse supervision, while the Compensation part aims to leverage 3D spatial and 2D semantic information to compensate for the information loss of previous disruptions. Extensive experimental results demonstrate that by reducing the impact of LDL, our framework with sparse supervision outperforms the state-of-the-art dense supervision methods with 11.6% improvement in Mean Absolute Error (MAE)} and 1.6x speedup in Frame Per Second (FPS)}. The code is available at https://github.com/megvii-research/Sparse-Beats-Dense.

Related papers

• RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation [14.466573808593887]
  RaCalNet is a novel framework that eliminates the need for dense supervision by using sparse LiDAR to supervise the learning of refined radar measurements.<n>RaCalNet produces depth maps with clear object contours and fine-grained textures, demonstrating superior visual quality compared to state-of-the-art dense-supervised methods.
  arXiv  Detail & Related papers  (2025-06-18T15:35:16Z)
• LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives [16.213116971476083]
  This review adopts a unified weakly supervised learning perspective to examine research on LiDAR interpretation and inversion. We summarize the latest advancements, provide a comprehensive review of the development and application of weakly supervised techniques in LiDAR remote sensing, and discuss potential future research directions in this field.
  arXiv  Detail & Related papers  (2025-03-24T06:51:38Z)
• Jasmine: Harnessing Diffusion Prior for Self-supervised Depth Estimation [55.501710766726234]
  Jasmine is a Stable Diffusion-based self-supervised framework for monocular depth estimation. It harnesses SD's visual priors to enhance the sharpness and generalization of unsupervised prediction. It achieves SoTA performance on the KITTI benchmark and exhibits superior zero-shot generalization across multiple datasets.
  arXiv  Detail & Related papers  (2025-03-20T07:15:49Z)
• Uncertainty-guided Optimal Transport in Depth Supervised Sparse-View 3D Gaussian [49.21866794516328]
  3D Gaussian splatting has demonstrated impressive performance in real-time novel view synthesis. Previous approaches have incorporated depth supervision into the training of 3D Gaussians to mitigate overfitting. We introduce a novel method to supervise the depth distribution of 3D Gaussians, utilizing depth priors with integrated uncertainty estimates.
  arXiv  Detail & Related papers  (2024-05-30T03:18:30Z)
• RIDERS: Radar-Infrared Depth Estimation for Robust Sensing [22.10378524682712]
  Adverse weather conditions pose significant challenges to accurate dense depth estimation. We present a novel approach for robust metric depth estimation by fusing a millimeter-wave Radar and a monocular infrared thermal camera. Our method achieves exceptional visual quality and accurate metric estimation by addressing the challenges of ambiguity and misalignment.
  arXiv  Detail & Related papers  (2024-02-03T07:14:43Z)
• LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection [96.63947479020631]
  In many real-world applications, the LiDAR points used by mass-produced robots and vehicles usually have fewer beams than that in large-scale public datasets. We propose the LiDAR Distillation to bridge the domain gap induced by different LiDAR beams for 3D object detection.
  arXiv  Detail & Related papers  (2022-03-28T17:59:02Z)
• Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR [22.202192422883122]
  We propose a novel two-stage network to advance the self-supervised monocular dense depth learning. Our model fuses monocular image features and sparse LiDAR features to predict initial depth maps. Our model outperforms the state-of-the-art sparse-LiDAR-based method (Pseudo-LiDAR++) by more than 68% for the downstream task monocular 3D object detection.
  arXiv  Detail & Related papers  (2021-09-20T15:28:36Z)
• Depth Estimation from Monocular Images and Sparse radar using Deep Ordinal Regression Network [2.0446891814677692]
  We integrate sparse radar data into a monocular depth estimation model and introduce a novel preprocessing method for reducing the sparseness and limited field of view provided by radar. We propose a novel method for estimating dense depth maps from monocular 2D images and sparse radar measurements using deep learning based on the deep ordinal regression network by Fu et al.
  arXiv  Detail & Related papers  (2021-07-15T20:17:48Z)
• Depth-supervised NeRF: Fewer Views and Faster Training for Free [69.34556647743285]
  DS-NeRF (Depth-supervised Neural Radiance Fields) is a loss for learning fields that takes advantage of readily-available depth supervision. We show that our loss is compatible with other recently proposed NeRF methods, demonstrating that depth is a cheap and easily digestible supervisory signal.
  arXiv  Detail & Related papers  (2021-07-06T17:58:35Z)
• LEAD: LiDAR Extender for Autonomous Driving [48.233424487002445]
  MEMS LiDAR emerges with irresistible trend due to its lower cost, more robust, and meeting the mass-production standards. It suffers small field of view (FoV), slowing down the step of its population. We propose LEAD, i.e., LiDAR Extender for Autonomous Driving, to extend the MEMS LiDAR by coupled image w.r.t both FoV and range.
  arXiv  Detail & Related papers  (2021-02-16T07:35:34Z)
• Unsupervised Object Detection with LiDAR Clues [70.73881791310495]
  We present the first practical method for unsupervised object detection with the aid of LiDAR clues. In our approach, candidate object segments based on 3D point clouds are firstly generated. Then, an iterative segment labeling process is conducted to assign segment labels and to train a segment labeling network. The labeling process is carefully designed so as to mitigate the issue of long-tailed and open-ended distribution.
  arXiv  Detail & Related papers  (2020-11-25T18:59:54Z)
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion [52.59664614744447]
  We present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous velocity measurements.
  arXiv  Detail & Related papers  (2020-10-02T00:13:00Z)
• Monocular Depth Prediction through Continuous 3D Loss [16.617016980396865]
  This paper reports a new continuous 3D loss function for learning depth from monocular images. The dense depth prediction from a monocular image is supervised using sparse LIDAR points. Experimental evaluation shows that the proposed loss improves the depth prediction accuracy and produces point-clouds with more consistent 3D geometric structures.
  arXiv  Detail & Related papers  (2020-03-21T22:47:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.