DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation

• URL: http://arxiv.org/abs/2406.16072v1
• Date: Sun, 23 Jun 2024 10:48:42 GMT
• Title: DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation
• Authors: Yueru Luo, Shuguang Cui, Zhen Li,
• Abstract summary: We present DV-3DLane, a novel end-to-end Dual-View multi-modal 3D Lane detection framework. It synergizes the strengths of both images and LiDAR points. It achieves state-of-the-art performance, with a remarkable 11.2 gain in F1 score and a substantial 53.5% reduction in errors.
• Score: 40.71071200694655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate 3D lane estimation is crucial for ensuring safety in autonomous driving. However, prevailing monocular techniques suffer from depth loss and lighting variations, hampering accurate 3D lane detection. In contrast, LiDAR points offer geometric cues and enable precise localization. In this paper, we present DV-3DLane, a novel end-to-end Dual-View multi-modal 3D Lane detection framework that synergizes the strengths of both images and LiDAR points. We propose to learn multi-modal features in dual-view spaces, i.e., perspective view (PV) and bird's-eye-view (BEV), effectively leveraging the modal-specific information. To achieve this, we introduce three designs: 1) A bidirectional feature fusion strategy that integrates multi-modal features into each view space, exploiting their unique strengths. 2) A unified query generation approach that leverages lane-aware knowledge from both PV and BEV spaces to generate queries. 3) A 3D dual-view deformable attention mechanism, which aggregates discriminative features from both PV and BEV spaces into queries for accurate 3D lane detection. Extensive experiments on the public benchmark, OpenLane, demonstrate the efficacy and efficiency of DV-3DLane. It achieves state-of-the-art performance, with a remarkable 11.2 gain in F1 score and a substantial 53.5% reduction in errors. The code is available at \url{https://github.com/JMoonr/dv-3dlane}.

Related papers

• OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagation [67.56268991234371]
  OV-Uni3DETR achieves the state-of-the-art performance on various scenarios, surpassing existing methods by more than 6% on average. Code and pre-trained models will be released later.
  arXiv  Detail & Related papers  (2024-03-28T17:05:04Z)
• Towards Unified 3D Object Detection via Algorithm and Data Unification [70.27631528933482]
  We build the first unified multi-modal 3D object detection benchmark MM- Omni3D and extend the aforementioned monocular detector to its multi-modal version. We name the designed monocular and multi-modal detectors as UniMODE and MM-UniMODE, respectively.
  arXiv  Detail & Related papers  (2024-02-28T18:59:31Z)
• Regulating Intermediate 3D Features for Vision-Centric Autonomous Driving [26.03800936700545]
  We propose to regulate intermediate dense 3D features with the help of volume rendering. Experimental results on the Occ3D and nuScenes datasets demonstrate that Vampire facilitates fine-grained and appropriate extraction of dense 3D features.
  arXiv  Detail & Related papers  (2023-12-19T04:09:05Z)
• UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving [47.590099762244535]
  Masked Autoencoders (MAE) play a pivotal role in learning potent representations, delivering outstanding results across various 3D perception tasks. This research delves into multi-modal Masked Autoencoders tailored for a unified representation space in autonomous driving. To intricately marry the semantics inherent in images with the geometric intricacies of LiDAR point clouds, we propose UniM$2$AE.
  arXiv  Detail & Related papers  (2023-08-21T02:13:40Z)
• An Efficient Transformer for Simultaneous Learning of BEV and Lane Representations in 3D Lane Detection [55.281369497158515]
  We propose an efficient transformer for 3D lane detection. Different from the vanilla transformer, our model contains a cross-attention mechanism to simultaneously learn lane and BEV representations. Our method obtains 2D and 3D lane predictions by applying the lane features to the image-view and BEV features, respectively.
  arXiv  Detail & Related papers  (2023-06-08T04:18:31Z)
• ONCE-3DLanes: Building Monocular 3D Lane Detection [41.46466150783367]
  We present ONCE-3DLanes, a real-world autonomous driving dataset with lane layout annotation in 3D space. By exploiting the explicit relationship between point clouds and image pixels, a dataset annotation pipeline is designed to automatically generate high-quality 3D lane locations.
  arXiv  Detail & Related papers  (2022-04-30T16:35:25Z)
• PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark [109.03773439461615]
  PersFormer is an end-to-end monocular 3D lane detector with a novel Transformer-based spatial feature transformation module. We release one of the first large-scale real-world 3D lane datasets, called OpenLane, with high-quality annotation and scenario diversity.
  arXiv  Detail & Related papers  (2022-03-21T16:12:53Z)
• Multi-modal 3D Human Pose Estimation with 2D Weak Supervision in Autonomous Driving [74.74519047735916]
  3D human pose estimation (HPE) in autonomous vehicles (AV) differs from other use cases in many factors. Data collected for other use cases (such as virtual reality, gaming, and animation) may not be usable for AV applications. We propose one of the first approaches to alleviate this problem in the AV setting.
  arXiv  Detail & Related papers  (2021-12-22T18:57:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.