SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation

• URL: http://arxiv.org/abs/2211.15656v3
• Date: Thu, 31 Oct 2024 15:01:41 GMT
• Title: SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation
• Authors: Hao Dong, Weihao Gu, Xianjing Zhang, Jintao Xu, Rui Ai, Huimin Lu, Juho Kannala, Xieyuanli Chen,
• Abstract summary: We propose a novel network named SuperFusion, exploiting the fusion of LiDAR and camera data at multiple levels. We benchmark our SuperFusion on the nuScenes dataset and a self-recorded dataset and show that it outperforms the state-of-the-art baseline methods.
• Score: 13.840020080021292
• License:
• Abstract: High-definition (HD) semantic map generation of the environment is an essential component of autonomous driving. Existing methods have achieved good performance in this task by fusing different sensor modalities, such as LiDAR and camera. However, current works are based on raw data or network feature-level fusion and only consider short-range HD map generation, limiting their deployment to realistic autonomous driving applications. In this paper, we focus on the task of building the HD maps in both short ranges, i.e., within 30 m, and also predicting long-range HD maps up to 90 m, which is required by downstream path planning and control tasks to improve the smoothness and safety of autonomous driving. To this end, we propose a novel network named SuperFusion, exploiting the fusion of LiDAR and camera data at multiple levels. We use LiDAR depth to improve image depth estimation and use image features to guide long-range LiDAR feature prediction. We benchmark our SuperFusion on the nuScenes dataset and a self-recorded dataset and show that it outperforms the state-of-the-art baseline methods with large margins on all intervals. Additionally, we apply the generated HD map to a downstream path planning task, demonstrating that the long-range HD maps predicted by our method can lead to better path planning for autonomous vehicles. Our code has been released at https://github.com/haomo-ai/SuperFusion.

Related papers

• DeepAerialMapper: Deep Learning-based Semi-automatic HD Map Creation for Highly Automated Vehicles [0.0]
  We introduce a semi-automatic method for creating HD maps from high-resolution aerial imagery. Our method involves training neural networks to semantically segment aerial images into classes relevant to HD maps. Exporting the map to the Lanelet2 format allows easy extension for different use cases.
  arXiv  Detail & Related papers  (2024-10-01T15:05:05Z)
• Driving with Prior Maps: Unified Vector Prior Encoding for Autonomous Vehicle Mapping [18.97422977086127]
  High-Definition Maps (HD maps) are essential for the precise navigation and decision-making of autonomous vehicles. The online construction of HD maps using on-board sensors has emerged as a promising solution. This paper proposes the PriorDrive framework to address these limitations by harnessing the power of prior maps.
  arXiv  Detail & Related papers  (2024-09-09T06:17:46Z)
• Augmenting Lane Perception and Topology Understanding with Standard Definition Navigation Maps [51.24861159115138]
  Standard Definition (SD) maps are more affordable and have worldwide coverage, offering a scalable alternative. We propose a novel framework to integrate SD maps into online map prediction and propose a Transformer-based encoder, SD Map Representations from transFormers. This enhancement consistently and significantly boosts (by up to 60%) lane detection and topology prediction on current state-of-the-art online map prediction methods.
  arXiv  Detail & Related papers  (2023-11-07T15:42:22Z)
• Prior Based Online Lane Graph Extraction from Single Onboard Camera Image [133.68032636906133]
  We tackle online estimation of the lane graph from a single onboard camera image. The prior is extracted from the dataset through a transformer based Wasserstein Autoencoder. The autoencoder is then used to enhance the initial lane graph estimates.
  arXiv  Detail & Related papers  (2023-07-25T08:58:26Z)
• HDMapNet: An Online HD Map Construction and Evaluation Framework [23.19001503634617]
  HD map construction is a crucial problem for autonomous driving. Traditional HD maps are coupled with centimeter-level accurate localization which is unreliable in many scenarios. Online map learning is a more scalable way to provide semantic and geometry priors to self-driving vehicles.
  arXiv  Detail & Related papers  (2021-07-13T18:06:46Z)
• HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps [81.86923212296863]
  HD maps are maps with precise definitions of road lanes with rich semantics of the traffic rules. There are only a small amount of real-world road topologies and geometries, which significantly limits our ability to test out the self-driving stack. We propose HDMapGen, a hierarchical graph generation model capable of producing high-quality and diverse HD maps.
  arXiv  Detail & Related papers  (2021-06-28T17:59:30Z)
• MP3: A Unified Model to Map, Perceive, Predict and Plan [84.07678019017644]
  MP3 is an end-to-end approach to mapless driving where the input is raw sensor data and a high-level command. We show that our approach is significantly safer, more comfortable, and can follow commands better than the baselines in challenging long-term closed-loop simulations.
  arXiv  Detail & Related papers  (2021-01-18T00:09:30Z)
• HDNET: Exploiting HD Maps for 3D Object Detection [99.49035895393934]
  We show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. We design a single stage detector that extracts geometric and semantic features from the HD maps. As maps might not be available everywhere, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data.
  arXiv  Detail & Related papers  (2020-12-21T21:59:54Z)
• Convolutional Recurrent Network for Road Boundary Extraction [99.55522995570063]
  We tackle the problem of drivable road boundary extraction from LiDAR and camera imagery. We design a structured model where a fully convolutional network obtains deep features encoding the location and direction of road boundaries. We showcase the effectiveness of our method on a large North American city where we obtain perfect topology of road boundaries 99.3% of the time.
  arXiv  Detail & Related papers  (2020-12-21T18:59:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.