U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based
Relocalization
- URL: http://arxiv.org/abs/2310.13766v1
- Date: Fri, 20 Oct 2023 18:57:38 GMT
- Title: U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based
Relocalization
- Authors: Andrea Boscolo Camiletto, Alfredo Bochicchio, Alexander Liniger,
Dengxin Dai, Abel Gawel
- Abstract summary: Relocalization is essential for intelligent vehicles when GPS reception is insufficient or sensor-based localization fails.
Recent advances in Bird's-Eye-View (BEV) segmentation allow for accurate estimation of local scene appearance.
This paper presents U-BEV, a U-Net inspired architecture that extends the current state-of-the-art by allowing the BEV to reason about the scene on multiple height layers before flattening the BEV features.
- Score: 86.63465798307728
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Efficient relocalization is essential for intelligent vehicles when GPS
reception is insufficient or sensor-based localization fails. Recent advances
in Bird's-Eye-View (BEV) segmentation allow for accurate estimation of local
scene appearance and in turn, can benefit the relocalization of the vehicle.
However, one downside of BEV methods is the heavy computation required to
leverage the geometric constraints. This paper presents U-BEV, a U-Net inspired
architecture that extends the current state-of-the-art by allowing the BEV to
reason about the scene on multiple height layers before flattening the BEV
features. We show that this extension boosts the performance of the U-BEV by up
to 4.11 IoU. Additionally, we combine the encoded neural BEV with a
differentiable template matcher to perform relocalization on neural SD-map
data. The model is fully end-to-end trainable and outperforms transformer-based
BEV methods of similar computational complexity by 1.7 to 2.8 mIoU and
BEV-based relocalization by over 26% Recall Accuracy on the nuScenes dataset.
Related papers
- BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight [30.45553559416835]
We propose BLOS-BEV, a novel BEV segmentation model that incorporates SD maps for accurate beyond line-of-sight perception, up to 200m.
Our approach is applicable to common BEV architectures and can achieve excellent results by incorporating information derived from SD maps.
arXiv Detail & Related papers (2024-07-11T14:15:48Z) - Window-to-Window BEV Representation Learning for Limited FoV Cross-View Geo-localization [17.388776062997813]
We propose a novel Window-to-Window BEV representation learning method, termed W2W-BEV, which adaptively matches BEV queries to ground reference at window-scale.
We demonstrate significant superiority of our W2W-BEV over previous state-of-the-art methods under challenging conditions of unknown orientation and limited FoV.
arXiv Detail & Related papers (2024-07-09T13:48:28Z) - Addressing Diverging Training Costs using Local Restoration for Precise Bird's Eye View Map Construction [24.897271474499767]
In this paper, we introduce our novel Trumpet Neural Network (TNN) mechanism.
The TNN provides a plug-and-play memory-efficient pipeline, thereby enabling the effective estimation of real-sized (or precise) semantic labels.
Our experiments show that the TNN mechanism provides a plug-and-play memory-efficient pipeline, thereby enabling the effective estimation of real-sized (or precise) semantic labels.
arXiv Detail & Related papers (2024-05-02T05:35:10Z) - RoadBEV: Road Surface Reconstruction in Bird's Eye View [55.0558717607946]
Vision-based online road reconstruction promisingly captures road information in advance.
Recent technique of Bird's-Eye-View (BEV) perception provides immense potential to more reliable and accurate reconstruction.
This paper uniformly proposes two simple yet effective models for road elevation reconstruction in BEV named RoadBEV-mono and RoadBEV-stereo.
arXiv Detail & Related papers (2024-04-09T20:24:29Z) - Improving Bird's Eye View Semantic Segmentation by Task Decomposition [42.57351039508863]
We decompose the original BEV segmentation task into two stages, namely BEV map reconstruction and RGB-BEV feature alignment.
Our approach simplifies the complexity of combining perception and generation into distinct steps, equipping the model to handle intricate and challenging scenes effectively.
arXiv Detail & Related papers (2024-04-02T13:19:45Z) - FB-BEV: BEV Representation from Forward-Backward View Transformations [131.11787050205697]
We propose a novel View Transformation Module (VTM) for Bird-Eye-View (BEV) representation.
We instantiate the proposed module with FB-BEV, which achieves a new state-of-the-art result of 62.4% NDS on the nuScenes test set.
arXiv Detail & Related papers (2023-08-04T10:26:55Z) - BEV-IO: Enhancing Bird's-Eye-View 3D Detection with Instance Occupancy [58.92659367605442]
We present BEV-IO, a new 3D detection paradigm to enhance BEV representation with instance occupancy information.
We show that BEV-IO can outperform state-of-the-art methods while only adding a negligible increase in parameters and computational overhead.
arXiv Detail & Related papers (2023-05-26T11:16:12Z) - MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation [104.12419434114365]
In real-world applications, sensor corruptions and failures lead to inferior performances.
We propose a robust framework, called MetaBEV, to address extreme real-world environments.
We show MetaBEV outperforms prior arts by a large margin on both full and corrupted modalities.
arXiv Detail & Related papers (2023-04-19T16:37:17Z) - GitNet: Geometric Prior-based Transformation for Birds-Eye-View
Segmentation [105.19949897812494]
Birds-eye-view (BEV) semantic segmentation is critical for autonomous driving.
We present a novel two-stage Geometry Prior-based Transformation framework named GitNet.
arXiv Detail & Related papers (2022-04-16T06:46:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.