RESAR-BEV: An Explainable Progressive Residual Autoregressive Approach for Camera-Radar Fusion in BEV Segmentation

• URL: http://arxiv.org/abs/2505.06515v1
• Date: Sat, 10 May 2025 05:10:07 GMT
• Title: RESAR-BEV: An Explainable Progressive Residual Autoregressive Approach for Camera-Radar Fusion in BEV Segmentation
• Abstract summary: Bird's-Eye-View (BEV) semantic segmentation provides comprehensive environmental perception for autonomous driving.<n>We propose RESAR-BEV, a progressive refinement framework that advances beyond single-step end-to-end approaches.<n> Experiments on nuScenes demonstrate RESAR-BEV state-of-the-art performance with 54.0% mIoU across 7 essential driving-scene categories.
• Score: 4.043972974168962
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bird's-Eye-View (BEV) semantic segmentation provides comprehensive environmental perception for autonomous driving but suffers multi-modal misalignment and sensor noise. We propose RESAR-BEV, a progressive refinement framework that advances beyond single-step end-to-end approaches: (1) progressive refinement through residual autoregressive learning that decomposes BEV segmentation into interpretable coarse-to-fine stages via our Drive-Transformer and Modifier-Transformer residual prediction cascaded architecture, (2) robust BEV representation combining ground-proximity voxels with adaptive height offsets and dual-path voxel feature encoding (max+attention pooling) for efficient feature extraction, and (3) decoupled supervision with offline Ground Truth decomposition and online joint optimization to prevent overfitting while ensuring structural coherence. Experiments on nuScenes demonstrate RESAR-BEV achieves state-of-the-art performance with 54.0% mIoU across 7 essential driving-scene categories while maintaining real-time capability at 14.6 FPS. The framework exhibits robustness in challenging scenarios of long-range perception and adverse weather conditions.

Related papers

• Variational Inference for Bird's Eye View Segmentation in Autonomous Driving [10.10734151875358]
  Bird's eye view (BEV) has emerged as a pivotal approach for environmental perception in autonomous driving.<n>We propose a novel transformer-based variational flow transformation network for BEV segmentation, denoted as TVB.<n>TVB implicitly learns the mapping from multiple camera views to a unified canonical BEV map during training by exploiting posterior BEV supervision.
  arXiv  Detail & Related papers  (2026-07-16T08:20:37Z)
• Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints [20.26519299938903]
  We introduce open-vocabulary BEV segmentation (OVBS) to recognize categories beyond the training set.<n>OVBEVSeg is a geometry-aware OVBS framework that enhances efficient Gaussian splatting (GS)-based unprojection.<n>On the nuScenes dataset, OVBEVSeg achieves state-of-the-art performance, outperforming closed-set methods by 15.3 mIoU on unseen categories.
  arXiv  Detail & Related papers  (2026-06-23T09:43:12Z)
• Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving [79.71689985927625]
  We propose IRR-Drive, an adaptive multimodal reflection framework for autonomous driving.<n>IRR-Drive explicitly models anticipated scene evolution, enabling the model to rigorously self-correct and refine its initial intent.<n>Our method achieves state-of-the-art performance on the benchmark in both PDMS and NAVSIMS.
  arXiv  Detail & Related papers  (2026-06-22T06:53:58Z)
• FRUC: Feedforward Dynamic Scene Reconstruction from Uncalibrated Collaborative Driving Views [19.310794789647222]
  We present FRUC, a feed-forward 3D Gaussian splatting framework for hindered dynamic scene reconstruction from und collaborative driving views.<n> FRUC is shown to be a new state-of-the-art for the scene reconstruction of dynamic collaborative driving environments, significantly outperforming existing methods in both rendering quality and efficiency.
  arXiv  Detail & Related papers  (2026-05-28T14:27:30Z)
• FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Driving [11.275815014211046]
  Building a learning-based closed-loop simulator for autonomous driving poses three major challenges.<n>Long-horizon temporal and cross-view consistency, mitigating autoregressive degradation under iterative self-conditioning, and satisfying low-latency inference constraints.<n>We propose FAR-Drive, a frame-level autoregressive video generation framework for autonomous driving.
  arXiv  Detail & Related papers  (2026-03-16T07:40:13Z)
• RESBev: Making BEV Perception More Robust [19.055695534533065]
  Real-world challenges from sensor deployment and adversarial attacks can cause severe anomalies and compromise the safety of autonomous driving systems.<n>We propose a resilient and plug-and-play BEV perception method, RESBev, which can be easily applied to existing BEV perception methods to enhance their robustness to diverse disturbances.<n>We show that RESBev significantly improves the robustness of existing BEV perception models against various external disturbances and adversarial attacks.
  arXiv  Detail & Related papers  (2026-03-10T11:36:52Z)
• Faster Training, Fewer Labels: Self-Supervised Pretraining for Fine-Grained BEV Segmentation [6.399280002773129]
  We propose a two-phase training strategy for fine-grained road marking segmentation.<n>During the self-supervised pretraining, BEVFormer predictions are differentiably reprojected into the image plane.<n>The subsequent supervised fine-tuning phase requires only 50% of the dataset and significantly less training time.
  arXiv  Detail & Related papers  (2026-02-20T08:37:58Z)
• Optimization-Guided Diffusion for Interactive Scene Generation [52.23368750264419]
  We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling.<n>We show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes.<n>Our approach can also generate $5times$ more near-collision frames with a time-to-collision under three seconds.
  arXiv  Detail & Related papers  (2025-12-08T15:56:18Z)
• Refine-and-Contrast: Adaptive Instance-Aware BEV Representations for Multi-UAV Collaborative Object Detection [15.494912154439367]
  Multi-UAV collaborative 3D detection enables accurate and robust perception by fusing multi-view observations from aerial platforms.<n>We present AdaBEV, a novel framework that learns adaptive instance-aware BEV representations through a refine-and-contrast paradigm.
  arXiv  Detail & Related papers  (2025-08-18T07:37:14Z)
• FastRSR: Efficient and Accurate Road Surface Reconstruction from Bird's Eye View [1.4999444543328293]
  Road Surface Reconstruction is crucial for autonomous driving.<n>Current methods for transforming perspective views to Bird's Eye View (BEV) face challenges such as information loss and representation sparsity.<n>To address these challenges, we propose two efficient and accurate BEV-based RSR models.
  arXiv  Detail & Related papers  (2025-04-13T11:48:52Z)
• End-to-End Driving with Online Trajectory Evaluation via BEV World Model [52.10633338584164]
  We propose an end-to-end driving framework WoTE, which leverages a BEV World model to predict future BEV states for Trajectory Evaluation.<n>We validate our framework on the NAVSIM benchmark and the closed-loop Bench2Drive benchmark based on the CARLA simulator, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-04-02T17:47:23Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• BEVMOSNet: Multimodal Fusion for BEV Moving Object Segmentation [3.613463012025065]
  We introduce BEVMOSNet, the first end-to-end multimodal fusion leveraging cameras, LiDAR, and radar to precisely predict the moving objects in bird's-eye-view (BEV)<n>We show an overall improvement in IoU score of 36.59% compared to the vision-based unimodal baseline BEV-MoSeg.
  arXiv  Detail & Related papers  (2025-03-05T09:03:46Z)
• OE-BevSeg: An Object Informed and Environment Aware Multimodal Framework for Bird's-eye-view Vehicle Semantic Segmentation [57.2213693781672]
  Bird's-eye-view (BEV) semantic segmentation is becoming crucial in autonomous driving systems. We propose OE-BevSeg, an end-to-end multimodal framework that enhances BEV segmentation performance. Our approach achieves state-of-the-art results by a large margin on the nuScenes dataset for vehicle segmentation.
  arXiv  Detail & Related papers  (2024-07-18T03:48:22Z)
• BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents [56.33989853438012]
  We propose BEVWorld, a framework that transforms multimodal sensor inputs into a unified and compact Bird's Eye View latent space for holistic environment modeling.<n>The proposed world model consists of two main components: a multi-modal tokenizer and a latent BEV sequence diffusion model.
  arXiv  Detail & Related papers  (2024-07-08T07:26:08Z)
• Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving [55.93813178692077]
  We present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms.<n>We assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction.<n>Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data.
  arXiv  Detail & Related papers  (2024-05-27T17:59:39Z)
• Towards Efficient 3D Object Detection in Bird's-Eye-View Space for Autonomous Driving: A Convolutional-Only Approach [13.513005108086006]
  We propose an efficient BEV-based 3D detection framework called BEVENet. BEVENet is 3$times$ faster than contemporary state-of-the-art (SOTA) approaches on the NuScenes challenge. Our experiments show that BEVENet is 3$times$ faster than contemporary state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2023-12-01T14:52:59Z)
• U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalization [81.76044207714637]
  Relocalization is essential for intelligent vehicles when GPS reception is insufficient or sensor-based localization fails.<n>Recent advances in Bird's-Eye-View (BEV) segmentation allow for accurate estimation of local scene appearance.<n>This paper presents U-BEV, a U-Net inspired architecture that extends the current state-of-the-art by allowing the BEV to reason about the scene on multiple height layers before flattening the BEV features.
  arXiv  Detail & Related papers  (2023-10-20T18:57:38Z)
• Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving [31.98600806479808]
  Bird's-Eye-View (BEV) representations have significantly improved the performance of 3D detectors with camera inputs on popular benchmarks. We evaluate the natural and adversarial robustness of various representative models under extensive settings. We propose a 3D consistent patch attack by applying adversarial patches in thetemporal 3D space to guarantee the consistency.
  arXiv  Detail & Related papers  (2023-03-30T11:16:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.