MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation

• URL: http://arxiv.org/abs/2407.11682v1
• Date: Tue, 16 Jul 2024 13:00:20 GMT
• Title: MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation
• Authors: Xiaoshuai Hao, Ruikai Li, Hui Zhang, Dingzhe Li, Rong Yin, Sangil Jung, Seung-In Park, ByungIn Yoo, Haimei Zhao, Jing Zhang,
• Abstract summary: We employ the Knowledge Distillation (KD) idea for efficient HD map construction for the first time. We introduce a novel KD-based approach called MapDistill to transfer knowledge from a high-performance camera-LiDAR fusion model to a lightweight camera-only model.
• Score: 13.057096630912952
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online high-definition (HD) map construction is an important and challenging task in autonomous driving. Recently, there has been a growing interest in cost-effective multi-view camera-based methods without relying on other sensors like LiDAR. However, these methods suffer from a lack of explicit depth information, necessitating the use of large models to achieve satisfactory performance. To address this, we employ the Knowledge Distillation (KD) idea for efficient HD map construction for the first time and introduce a novel KD-based approach called MapDistill to transfer knowledge from a high-performance camera-LiDAR fusion model to a lightweight camera-only model. Specifically, we adopt the teacher-student architecture, i.e., a camera-LiDAR fusion model as the teacher and a lightweight camera model as the student, and devise a dual BEV transform module to facilitate cross-modal knowledge distillation while maintaining cost-effective camera-only deployment. Additionally, we present a comprehensive distillation scheme encompassing cross-modal relation distillation, dual-level feature distillation, and map head distillation. This approach alleviates knowledge transfer challenges between modalities, enabling the student model to learn improved feature representations for HD map construction. Experimental results on the challenging nuScenes dataset demonstrate the effectiveness of MapDistill, surpassing existing competitors by over 7.7 mAP or 4.5X speedup.

Related papers

• KD360-VoxelBEV: LiDAR and 360-degree Camera Cross Modality Knowledge Distillation for Bird's-Eye-View Segmentation [30.730703237135216]
  We present the first cross-modality distillation framework specifically tailored for single-panoramic-camera Bird's-Eye-View (BEV) segmentation.<n>Our approach leverages a novel LiDAR image representation fused from range, intensity and ambient channels, together with a voxel-aligned view transformer.<n>During training, a high-capacity LiDAR and camera fusion Teacher network extracts both rich spatial and semantic features for cross-modality knowledge distillation.
  arXiv  Detail & Related papers  (2025-12-17T11:00:00Z)
• MapKD: Unlocking Prior Knowledge with Cross-Modal Distillation for Efficient Online HD Map Construction [23.156125781601528]
  MapKD is a novel multi-level cross-modal knowledge distillation framework with an innovative Teacher-Coach-Student (TCS) paradigm.<n>We introduce two targeted knowledge distillation strategies: Token-Guided 2D Patch Distillation (TGPD) for bird's eye view feature alignment and Masked Semantic Response Distillation (MSRD) for semantic learning guidance.<n>Experiments on the challenging nuScenes dataset demonstrate that MapKD improves the student model by +6.68 mIoU and +10.94 mAP while simultaneously accelerating inference speed.
  arXiv  Detail & Related papers  (2025-08-21T15:37:18Z)
• BridgeTA: Bridging the Representation Gap in Knowledge Distillation via Teacher Assistant for Bird's Eye View Map Segmentation [17.072492774587456]
  Camera-only approaches have drawn attention as cost-effective alternatives to LiDAR, but they still fall behind LiDAR-Camera (LC) fusion-based methods.<n>We introduce BridgeTA, a cost-effective distillation framework to bridge the representation gap between LC fusion and Camera-only models.<n>Our method achieves an improvement of 4.2% mIoU over the Camera-only baseline, up to 45% higher than the improvement of other state-of-the-art KD methods.
  arXiv  Detail & Related papers  (2025-08-13T08:28:21Z)
• What Really Matters for Robust Multi-Sensor HD Map Construction? [9.108124985480046]
  High-definition (HD) map construction methods are crucial for providing precise and comprehensive static environmental information.<n>Existing approaches primarily focus on improving model accuracy and often neglect the robustness of perception models.<n>We propose strategies to enhance the robustness of multi-modal fusion methods for HD map construction while maintaining high accuracy.
  arXiv  Detail & Related papers  (2025-07-02T08:46:27Z)
• MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation [17.27883003990266]
  Vision-and-Language Navigation (VLN) is a core task in Embodied AI. This paper introduces a two-stage knowledge distillation framework, producing a student model, MiniVLN. Our findings indicate that the two-stage distillation approach is more effective in narrowing the performance gap between the teacher model and the student model.
  arXiv  Detail & Related papers  (2024-09-27T14:54:54Z)
• Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models [62.5501109475725]
  Knowledge distillation (KD) is a technique that compresses large teacher models by training smaller student models to mimic them. This paper introduces Online Knowledge Distillation (OKD), where the teacher network integrates small online modules to concurrently train with the student model. OKD achieves or exceeds the performance of leading methods in various model architectures and sizes, reducing training time by up to fourfold.
  arXiv  Detail & Related papers  (2024-09-19T07:05:26Z)
• MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection [43.67544474555326]
  We introduce Monocular Teaching Assistant Knowledge Distillation (MonoTAKD) to transfer robust 3D visual knowledge to a camera-based student model. Experimental results show that our MonoTAKD achieves state-of-the-art performance on the KITTI3D dataset.
  arXiv  Detail & Related papers  (2024-04-07T10:39:04Z)
• TSCM: A Teacher-Student Model for Vision Place Recognition Using Cross-Metric Knowledge Distillation [6.856317526681759]
  Visual place recognition plays a pivotal role in autonomous exploration and navigation of mobile robots. Existing methods overcome this by exploiting powerful yet large networks. We propose a high-performance teacher and lightweight student distillation framework called TSCM.
  arXiv  Detail & Related papers  (2024-04-02T02:29:41Z)
• One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation [69.65734716679925]
  Knowledge distillation has proven to be a highly effective approach for enhancing model performance through a teacher-student training scheme. Most existing distillation methods are designed under the assumption that the teacher and student models belong to the same model family. We propose a simple yet effective one-for-all KD framework called OFA-KD, which significantly improves the distillation performance between heterogeneous architectures.
  arXiv  Detail & Related papers  (2023-10-30T11:13:02Z)
• Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection [66.74183705987276]
  We introduce a framework to improve the camera-only apprentice model, including an apprentice-friendly multi-modal expert and temporal-fusion-friendly distillation supervision. With those improvements, our camera-only apprentice VCD-A sets new state-of-the-art on nuScenes with a score of 63.1% NDS.
  arXiv  Detail & Related papers  (2023-10-24T09:29:26Z)
• DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillation [25.933070263556374]
  3D perception based on representations learned from multi-camera bird's-eye-view (BEV) is trending as cameras are cost-effective for mass production in autonomous driving industry. There exists a distinct performance gap between multi-camera BEV and LiDAR based 3D object detection. We propose to boost the representation learning of a multi-camera BEV based student detector by training it to imitate the features of a well-trained LiDAR based teacher detector.
  arXiv  Detail & Related papers  (2023-09-26T17:56:21Z)
• Cross Architecture Distillation for Face Recognition [49.55061794917994]
  We develop an Adaptable Prompting Teacher network (APT) that integrates prompts into the teacher, enabling it to manage distillation-specific knowledge. Experiments on popular face benchmarks and two large-scale verification sets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2023-06-26T12:54:28Z)
• SimDistill: Simulated Multi-modal Distillation for BEV 3D Object Detection [56.24700754048067]
  Multi-view camera-based 3D object detection has become popular due to its low cost, but accurately inferring 3D geometry solely from camera data remains challenging. We propose a Simulated multi-modal Distillation (SimDistill) method by carefully crafting the model architecture and distillation strategy. Our SimDistill can learn better feature representations for 3D object detection while maintaining a cost-effective camera-only deployment.
  arXiv  Detail & Related papers  (2023-03-29T16:08:59Z)
• Knowledge Distillation for 6D Pose Estimation by Keypoint Distribution Alignment [77.70208382044355]
  We introduce the first knowledge distillation method for 6D pose estimation. We observe the compact student network to struggle predicting precise 2D keypoint locations. Our experiments on several benchmarks show that our distillation method yields state-of-the-art results.
  arXiv  Detail & Related papers  (2022-05-30T10:17:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.