MV-Map: Offboard HD-Map Generation with Multi-view Consistency
- URL: http://arxiv.org/abs/2305.08851v3
- Date: Mon, 9 Oct 2023 01:15:32 GMT
- Title: MV-Map: Offboard HD-Map Generation with Multi-view Consistency
- Authors: Ziyang Xie and Ziqi Pang and Yu-Xiong Wang
- Abstract summary: Bird's-eye-view (BEV) perception models can be useful for building high-definition maps (HD-Maps) with less human labor.
Their results are often unreliable and demonstrate noticeable inconsistencies in the predicted HD-Maps from different viewpoints.
This paper advocates a more practical 'offboard' HD-Map generation setup that removes the computation constraints.
- Score: 29.797769409113105
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While bird's-eye-view (BEV) perception models can be useful for building
high-definition maps (HD-Maps) with less human labor, their results are often
unreliable and demonstrate noticeable inconsistencies in the predicted HD-Maps
from different viewpoints. This is because BEV perception is typically set up
in an 'onboard' manner, which restricts the computation and consequently
prevents algorithms from reasoning multiple views simultaneously. This paper
overcomes these limitations and advocates a more practical 'offboard' HD-Map
generation setup that removes the computation constraints, based on the fact
that HD-Maps are commonly reusable infrastructures built offline in data
centers. To this end, we propose a novel offboard pipeline called MV-Map that
capitalizes multi-view consistency and can handle an arbitrary number of frames
with the key design of a 'region-centric' framework. In MV-Map, the target
HD-Maps are created by aggregating all the frames of onboard predictions,
weighted by the confidence scores assigned by an 'uncertainty network'. To
further enhance multi-view consistency, we augment the uncertainty network with
the global 3D structure optimized by a voxelized neural radiance field
(Voxel-NeRF). Extensive experiments on nuScenes show that our MV-Map
significantly improves the quality of HD-Maps, further highlighting the
importance of offboard methods for HD-Map generation.
Related papers
- Progressive Query Refinement Framework for Bird's-Eye-View Semantic Segmentation from Surrounding Images [3.495246564946556]
We introduce the Multi-Resolution (MR) concept into Bird's-Eye-View (BEV) semantic segmentation for autonomous driving.
We propose a visual feature interaction network that promotes interactions between features across images and across feature levels.
We evaluate our model on a large-scale real-world dataset.
arXiv Detail & Related papers (2024-07-24T05:00:31Z) - ADMap: Anti-disturbance framework for reconstructing online vectorized
HD map [9.218463154577616]
This paper proposes the Anti-disturbance Map reconstruction framework (ADMap)
To mitigate point-order jitter, the framework consists of three modules: Multi-Scale Perception Neck, Instance Interactive Attention (IIA), and Vector Direction Difference Loss (VDDL)
arXiv Detail & Related papers (2024-01-24T01:37:27Z) - Online Map Vectorization for Autonomous Driving: A Rasterization
Perspective [58.71769343511168]
We introduce a newization-based evaluation metric, which has superior sensitivity and is better suited to real-world autonomous driving scenarios.
We also propose MapVR (Map Vectorization via Rasterization), a novel framework that applies differentiableization to preciseized outputs and then performs geometry-aware supervision on HD maps.
arXiv Detail & Related papers (2023-06-18T08:51:14Z) - NeMO: Neural Map Growing System for Spatiotemporal Fusion in
Bird's-Eye-View and BDD-Map Benchmark [9.430779563669908]
Vision-centric Bird's-Eye View representation is essential for autonomous driving systems.
This work outlines a new paradigm, named NeMO, for generating local maps through the utilization of a readable and writable big map.
With an assumption that the feature distribution of all BEV grids follows an identical pattern, we adopt a shared-weight neural network for all grids to update the big map.
arXiv Detail & Related papers (2023-06-07T15:46:15Z) - InstaGraM: Instance-level Graph Modeling for Vectorized HD Map Learning [6.062751776009753]
We propose online HD map learning framework that detects HD map elements from onboard sensor observations.
InstaGraM, instance-level graph modeling of HD map brings accurate and fast end-to-end vectorized HD map learning.
Our proposed network outperforms previous models by up to 13.7 mAP with up to 33.8X faster time.
arXiv Detail & Related papers (2023-01-10T08:15:35Z) - BEVBert: Multimodal Map Pre-training for Language-guided Navigation [75.23388288113817]
We propose a new map-based pre-training paradigm that is spatial-aware for use in vision-and-language navigation (VLN)
We build a local metric map to explicitly aggregate incomplete observations and remove duplicates, while modeling navigation dependency in a global topological map.
Based on the hybrid map, we devise a pre-training framework to learn a multimodal map representation, which enhances spatial-aware cross-modal reasoning thereby facilitating the language-guided navigation goal.
arXiv Detail & Related papers (2022-12-08T16:27:54Z) - Monocular BEV Perception of Road Scenes via Front-to-Top View Projection [57.19891435386843]
We present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view.
Our model runs at 25 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction.
arXiv Detail & Related papers (2022-11-15T13:52:41Z) - TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view
Stereo [55.30992853477754]
We present TANDEM, a real-time monocular tracking and dense framework.
For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of alignments.
TANDEM shows state-of-the-art real-time 3D reconstruction performance.
arXiv Detail & Related papers (2021-11-14T19:01:02Z) - HDMapNet: An Online HD Map Construction and Evaluation Framework [23.19001503634617]
HD map construction is a crucial problem for autonomous driving.
Traditional HD maps are coupled with centimeter-level accurate localization which is unreliable in many scenarios.
Online map learning is a more scalable way to provide semantic and geometry priors to self-driving vehicles.
arXiv Detail & Related papers (2021-07-13T18:06:46Z) - HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps [81.86923212296863]
HD maps are maps with precise definitions of road lanes with rich semantics of the traffic rules.
There are only a small amount of real-world road topologies and geometries, which significantly limits our ability to test out the self-driving stack.
We propose HDMapGen, a hierarchical graph generation model capable of producing high-quality and diverse HD maps.
arXiv Detail & Related papers (2021-06-28T17:59:30Z) - HDNET: Exploiting HD Maps for 3D Object Detection [99.49035895393934]
We show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors.
We design a single stage detector that extracts geometric and semantic features from the HD maps.
As maps might not be available everywhere, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data.
arXiv Detail & Related papers (2020-12-21T21:59:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.