MapKD: Unlocking Prior Knowledge with Cross-Modal Distillation for Efficient Online HD Map Construction

• URL: http://arxiv.org/abs/2508.15653v2
• Date: Fri, 22 Aug 2025 01:44:26 GMT
• Title: MapKD: Unlocking Prior Knowledge with Cross-Modal Distillation for Efficient Online HD Map Construction
• Authors: Ziyang Yan, Ruikai Li, Zhiyong Cui, Bohan Li, Han Jiang, Yilong Ren, Aoyong Li, Zhenning Li, Sijia Wen, Haiyang Yu,
• Abstract summary: MapKD is a novel multi-level cross-modal knowledge distillation framework with an innovative Teacher-Coach-Student (TCS) paradigm.<n>We introduce two targeted knowledge distillation strategies: Token-Guided 2D Patch Distillation (TGPD) for bird's eye view feature alignment and Masked Semantic Response Distillation (MSRD) for semantic learning guidance.<n>Experiments on the challenging nuScenes dataset demonstrate that MapKD improves the student model by +6.68 mIoU and +10.94 mAP while simultaneously accelerating inference speed.
• Score: 23.156125781601528
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Online HD map construction is a fundamental task in autonomous driving systems, aiming to acquire semantic information of map elements around the ego vehicle based on real-time sensor inputs. Recently, several approaches have achieved promising results by incorporating offline priors such as SD maps and HD maps or by fusing multi-modal data. However, these methods depend on stale offline maps and multi-modal sensor suites, resulting in avoidable computational overhead at inference. To address these limitations, we employ a knowledge distillation strategy to transfer knowledge from multimodal models with prior knowledge to an efficient, low-cost, and vision-centric student model. Specifically, we propose MapKD, a novel multi-level cross-modal knowledge distillation framework with an innovative Teacher-Coach-Student (TCS) paradigm. This framework consists of: (1) a camera-LiDAR fusion model with SD/HD map priors serving as the teacher; (2) a vision-centric coach model with prior knowledge and simulated LiDAR to bridge the cross-modal knowledge transfer gap; and (3) a lightweight vision-based student model. Additionally, we introduce two targeted knowledge distillation strategies: Token-Guided 2D Patch Distillation (TGPD) for bird's eye view feature alignment and Masked Semantic Response Distillation (MSRD) for semantic learning guidance. Extensive experiments on the challenging nuScenes dataset demonstrate that MapKD improves the student model by +6.68 mIoU and +10.94 mAP while simultaneously accelerating inference speed. The code is available at:https://github.com/2004yan/MapKD2026.

Related papers

• What Really Matters for Robust Multi-Sensor HD Map Construction? [9.108124985480046]
  High-definition (HD) map construction methods are crucial for providing precise and comprehensive static environmental information.<n>Existing approaches primarily focus on improving model accuracy and often neglect the robustness of perception models.<n>We propose strategies to enhance the robustness of multi-modal fusion methods for HD map construction while maintaining high accuracy.
  arXiv  Detail & Related papers  (2025-07-02T08:46:27Z)
• JointDistill: Adaptive Multi-Task Distillation for Joint Depth Estimation and Scene Segmentation [31.89422375115854]
  This work explores how the multi-task distillation could be used to improve unified modeling.<n>We propose a self-adaptive distillation method that can adjust the knowledge amount from each teacher according to the student's current learning ability.<n>We evaluate our method on multiple benchmarking datasets including Cityscapes and NYU-v2.
  arXiv  Detail & Related papers  (2025-05-15T08:00:48Z)
• Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model [11.010846827099936]
  Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks.<n>We propose a method called Align-KD to guide the student model to learn the cross-modal matching that occurs at the shallow layer.<n>Under the guidance of Align-KD, the 1.7B MobileVLM V2 model can learn rich knowledge from the 7B teacher model with light design of training loss.
  arXiv  Detail & Related papers  (2024-12-02T08:55:19Z)
• TopoSD: Topology-Enhanced Lane Segment Perception with SDMap Prior [70.84644266024571]
  We propose to train a perception model to "see" standard definition maps (SDMaps) We encode SDMap elements into neural spatial map representations and instance tokens, and then incorporate such complementary features as prior information. Based on the lane segment representation framework, the model simultaneously predicts lanes, centrelines and their topology.
  arXiv  Detail & Related papers  (2024-11-22T06:13:42Z)
• Multi Teacher Privileged Knowledge Distillation for Multimodal Expression Recognition [58.41784639847413]
  Human emotion is a complex phenomenon conveyed and perceived through facial expressions, vocal tones, body language, and physiological signals. In this paper, a multi-teacher PKD (MT-PKDOT) method with self-distillation is introduced to align diverse teacher representations before distilling them to the student. Results indicate that our proposed method can outperform SOTA PKD methods.
  arXiv  Detail & Related papers  (2024-08-16T22:11:01Z)
• MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation [13.057096630912952]
  We employ the Knowledge Distillation (KD) idea for efficient HD map construction for the first time. We introduce a novel KD-based approach called MapDistill to transfer knowledge from a high-performance camera-LiDAR fusion model to a lightweight camera-only model.
  arXiv  Detail & Related papers  (2024-07-16T13:00:20Z)
• HDKD: Hybrid Data-Efficient Knowledge Distillation Network for Medical Image Classification [0.0]
  Vision Transformers (ViTs) have achieved significant advancement in computer vision tasks due to their powerful modeling capacity.<n>Previous approaches to Knowledge Distillation (KD) have pursued two primary paths: some focused solely on distilling the logit distribution from CNN teacher to ViT student.<n>This paper presents Hybrid Data-efficient Knowledge Distillation (HDKD) paradigm which employs a CNN teacher and a hybrid student.
  arXiv  Detail & Related papers  (2024-07-10T10:09:12Z)
• Cross Architecture Distillation for Face Recognition [49.55061794917994]
  We develop an Adaptable Prompting Teacher network (APT) that integrates prompts into the teacher, enabling it to manage distillation-specific knowledge. Experiments on popular face benchmarks and two large-scale verification sets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2023-06-26T12:54:28Z)
• SimDistill: Simulated Multi-modal Distillation for BEV 3D Object Detection [56.24700754048067]
  Multi-view camera-based 3D object detection has become popular due to its low cost, but accurately inferring 3D geometry solely from camera data remains challenging. We propose a Simulated multi-modal Distillation (SimDistill) method by carefully crafting the model architecture and distillation strategy. Our SimDistill can learn better feature representations for 3D object detection while maintaining a cost-effective camera-only deployment.
  arXiv  Detail & Related papers  (2023-03-29T16:08:59Z)
• CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation [130.08432609780374]
  In 3D action recognition, there exists rich complementary information between skeleton modalities. We propose a new Cross-modal Mutual Distillation (CMD) framework with the following designs. Our approach outperforms existing self-supervised methods and sets a series of new records.
  arXiv  Detail & Related papers  (2022-08-26T06:06:09Z)
• Inter-Region Affinity Distillation for Road Marking Segmentation [81.3619453527367]
  We study the problem of distilling knowledge from a large deep teacher network to a much smaller student network. Our method is known as Inter-Region Affinity KD (IntRA-KD)
  arXiv  Detail & Related papers  (2020-04-11T04:26:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.