BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance

• URL: http://arxiv.org/abs/2502.19694v2
• Date: Mon, 24 Mar 2025 22:27:08 GMT
• Title: BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance
• Authors: Xin Ye, Burhaneddin Yaman, Sheng Cheng, Feng Tao, Abhirup Mallik, Liu Ren,
• Abstract summary: Bird's-eye-view (BEV) representations play a crucial role in autonomous driving tasks.<n> inherent noise, stemming from sensor limitations and the learning process, remains largely unaddressed.<n>We propose BEVDiffuser, a novel diffusion model that effectively denoises BEV feature maps using the ground-truth object layout as guidance.
• Score: 14.315057684079397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bird's-eye-view (BEV) representations play a crucial role in autonomous driving tasks. Despite recent advancements in BEV generation, inherent noise, stemming from sensor limitations and the learning process, remains largely unaddressed, resulting in suboptimal BEV representations that adversely impact the performance of downstream tasks. To address this, we propose BEVDiffuser, a novel diffusion model that effectively denoises BEV feature maps using the ground-truth object layout as guidance. BEVDiffuser can be operated in a plug-and-play manner during training time to enhance existing BEV models without requiring any architectural modifications. Extensive experiments on the challenging nuScenes dataset demonstrate BEVDiffuser's exceptional denoising and generation capabilities, which enable significant enhancement to existing BEV models, as evidenced by notable improvements of 12.3\% in mAP and 10.1\% in NDS achieved for 3D object detection without introducing additional computational complexity. Moreover, substantial improvements in long-tail object detection and under challenging weather and lighting conditions further validate BEVDiffuser's effectiveness in denoising and enhancing BEV representations.

Related papers

• Robust Bird's Eye View Segmentation by Adapting DINOv2 [3.236198583140341]
  We adapt a vision foundational model, DINOv2, to BEV estimation using Low Rank Adaptation (LoRA) Our experiments show increased robustness of BEV perception under various corruptions. We also showcase the effectiveness of the adapted representations in terms of fewer learnable parameters and faster convergence during training.
  arXiv  Detail & Related papers  (2024-09-16T12:23:35Z)
• FSD-BEV: Foreground Self-Distillation for Multi-view 3D Object Detection [33.225938984092274]
  We propose a Foreground Self-Distillation (FSD) scheme that effectively avoids the issue of distribution discrepancies. We also design two Point Cloud Intensification ( PCI) strategies to compensate for the sparsity of point clouds. We develop a Multi-Scale Foreground Enhancement (MSFE) module to extract and fuse multi-scale foreground features.
  arXiv  Detail & Related papers  (2024-07-14T09:39:44Z)
• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
  We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms.<n>We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction.<n>Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
  arXiv  Detail & Related papers  (2024-05-27T17:59:39Z)
• DA-BEV: Unsupervised Domain Adaptation for Bird's Eye View Perception [104.87876441265593]
  Camera-only Bird's Eye View (BEV) has demonstrated great potential in environment perception in a 3D space. Unsupervised domain adaptive BEV, which effective learning from various unlabelled target data, is far under-explored. We design DA-BEV, the first domain adaptive camera-only BEV framework that addresses domain adaptive BEV challenges by exploiting the complementary nature of image-view features and BEV features.
  arXiv  Detail & Related papers  (2024-01-13T04:21:24Z)
• Diffusion-Based Particle-DETR for BEV Perception [94.88305708174796]
  Bird-Eye-View (BEV) is one of the most widely-used scene representations for visual perception in Autonomous Vehicles (AVs) Recent diffusion-based methods offer a promising approach to uncertainty modeling for visual perception but fail to effectively detect small objects in the large coverage of the BEV. Here, we address this problem by combining the diffusion paradigm with current state-of-the-art 3D object detectors in BEV.
  arXiv  Detail & Related papers  (2023-12-18T09:52:14Z)
• U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalization [81.76044207714637]
  Relocalization is essential for intelligent vehicles when GPS reception is insufficient or sensor-based localization fails. Recent advances in Bird's-Eye-View (BEV) segmentation allow for accurate estimation of local scene appearance. This paper presents U-BEV, a U-Net inspired architecture that extends the current state-of-the-art by allowing the BEV to reason about the scene on multiple height layers before flattening the BEV features.
  arXiv  Detail & Related papers  (2023-10-20T18:57:38Z)
• QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection [57.019527599167255]
  Multi-view 3D detection based on BEV (bird-eye-view) has recently achieved significant improvements. We show in our paper that directly applying quantization in BEV tasks will 1) make the training unstable, and 2) lead to intolerable performance degradation. Our method QD-BEV enables a novel view-guided distillation (VGD) objective, which can stabilize the quantization-aware training (QAT) while enhancing the model performance.
  arXiv  Detail & Related papers  (2023-08-21T07:06:49Z)
• Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving [31.98600806479808]
  Bird's-Eye-View (BEV) representations have significantly improved the performance of 3D detectors with camera inputs on popular benchmarks. We evaluate the natural and adversarial robustness of various representative models under extensive settings. We propose a 3D consistent patch attack by applying adversarial patches in thetemporal 3D space to guarantee the consistency.
  arXiv  Detail & Related papers  (2023-03-30T11:16:58Z)
• DiffBEV: Conditional Diffusion Model for Bird's Eye View Perception [14.968177102647783]
  We propose an end-to-end framework, named DiffBEV, to exploit the potential of diffusion model to generate a more comprehensive BEV representation. In practice, we design three types of conditions to guide the training of the diffusion model which denoises the coarse samples and refines the semantic feature. We show that DiffBEV achieves a 25.9% mIoU on the nuScenes dataset, which is 6.2% higher than the best-performing existing approach.
  arXiv  Detail & Related papers  (2023-03-15T02:42:48Z)
• Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
  We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process. The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
  arXiv  Detail & Related papers  (2021-04-22T12:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.