U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalization

• URL: http://arxiv.org/abs/2310.13766v2
• Date: Sun, 1 Sep 2024 22:05:52 GMT
• Title: U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalization
• Authors: Andrea Boscolo Camiletto, Alfredo Bochicchio, Alexander Liniger, Dengxin Dai, Abel Gawel,
• Abstract summary: Relocalization is essential for intelligent vehicles when GPS reception is insufficient or sensor-based localization fails. Recent advances in Bird's-Eye-View (BEV) segmentation allow for accurate estimation of local scene appearance. This paper presents U-BEV, a U-Net inspired architecture that extends the current state-of-the-art by allowing the BEV to reason about the scene on multiple height layers before flattening the BEV features.
• Score: 81.76044207714637
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Efficient relocalization is essential for intelligent vehicles when GPS reception is insufficient or sensor-based localization fails. Recent advances in Bird's-Eye-View (BEV) segmentation allow for accurate estimation of local scene appearance and in turn, can benefit the relocalization of the vehicle. However, one downside of BEV methods is the heavy computation required to leverage the geometric constraints. This paper presents U-BEV, a U-Net inspired architecture that extends the current state-of-the-art by allowing the BEV to reason about the scene on multiple height layers before flattening the BEV features. We show that this extension boosts the performance of the U-BEV by up to 4.11 IoU. Additionally, we combine the encoded neural BEV with a differentiable template matcher to perform relocalization on neural SD-map data. The model is fully end-to-end trainable and outperforms transformer-based BEV methods of similar computational complexity by 1.7 to 2.8 mIoU and BEV-based relocalization by over 26% Recall Accuracy on the nuScenes dataset.

Related papers

• VQ-Map: Bird's-Eye-View Map Layout Estimation in Tokenized Discrete Space via Vector Quantization [108.68014173017583]
  Bird's-eye-view (BEV) map layout estimation requires an accurate and full understanding of the semantics for the environmental elements around the ego car. We propose to utilize a generative model similar to the Vector Quantized-Variational AutoEncoder (VQ-VAE) to acquire prior knowledge for the high-level BEV semantics in the tokenized discrete space. Thanks to the obtained BEV tokens accompanied with a codebook embedding encapsulating the semantics for different BEV elements in the groundtruth maps, we are able to directly align the sparse backbone image features with the obtained BEV tokens
  arXiv  Detail & Related papers  (2024-11-03T16:09:47Z)
• BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight [30.45553559416835]
  We propose BLOS-BEV, a novel BEV segmentation model that incorporates SD maps for accurate beyond line-of-sight perception, up to 200m. Our approach is applicable to common BEV architectures and can achieve excellent results by incorporating information derived from SD maps.
  arXiv  Detail & Related papers  (2024-07-11T14:15:48Z)
• Window-to-Window BEV Representation Learning for Limited FoV Cross-View Geo-localization [17.388776062997813]
  We propose a novel Window-to-Window BEV representation learning method, termed W2W-BEV, which adaptively matches BEV queries to ground reference at window-scale. We demonstrate significant superiority of our W2W-BEV over previous state-of-the-art methods under challenging conditions of unknown orientation and limited FoV.
  arXiv  Detail & Related papers  (2024-07-09T13:48:28Z)
• Improving Bird's Eye View Semantic Segmentation by Task Decomposition [42.57351039508863]
  We decompose the original BEV segmentation task into two stages, namely BEV map reconstruction and RGB-BEV feature alignment. Our approach simplifies the complexity of combining perception and generation into distinct steps, equipping the model to handle intricate and challenging scenes effectively.
  arXiv  Detail & Related papers  (2024-04-02T13:19:45Z)
• FB-BEV: BEV Representation from Forward-Backward View Transformations [131.11787050205697]
  We propose a novel View Transformation Module (VTM) for Bird-Eye-View (BEV) representation. We instantiate the proposed module with FB-BEV, which achieves a new state-of-the-art result of 62.4% NDS on the nuScenes test set.
  arXiv  Detail & Related papers  (2023-08-04T10:26:55Z)
• BEV-IO: Enhancing Bird's-Eye-View 3D Detection with Instance Occupancy [58.92659367605442]
  We present BEV-IO, a new 3D detection paradigm to enhance BEV representation with instance occupancy information. We show that BEV-IO can outperform state-of-the-art methods while only adding a negligible increase in parameters and computational overhead.
  arXiv  Detail & Related papers  (2023-05-26T11:16:12Z)
• MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation [104.12419434114365]
  In real-world applications, sensor corruptions and failures lead to inferior performances. We propose a robust framework, called MetaBEV, to address extreme real-world environments. We show MetaBEV outperforms prior arts by a large margin on both full and corrupted modalities.
  arXiv  Detail & Related papers  (2023-04-19T16:37:17Z)
• GitNet: Geometric Prior-based Transformation for Birds-Eye-View Segmentation [105.19949897812494]
  Birds-eye-view (BEV) semantic segmentation is critical for autonomous driving. We present a novel two-stage Geometry Prior-based Transformation framework named GitNet.
  arXiv  Detail & Related papers  (2022-04-16T06:46:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.