Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation

• URL: http://arxiv.org/abs/2509.00379v1
• Date: Sat, 30 Aug 2025 06:34:39 GMT
• Title: Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation
• Authors: Jialiang Kang, Jiawen Wang, Dingsheng Luo,
• Abstract summary: Traditional approaches rely on extensive data for annotated point cloud analysis.<n>Realworld image datasets offer abundant availability and substantial scale.<n>We propose two crossmodal knowledge distillation methods: Unsupervised Domain Knowledge Distillation (UDAKD) and Feature and Semantic-based Knowledge Distillation (FSKD)
• Score: 3.6903284283843942
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Semantic segmentation of 3D LiDAR data plays a pivotal role in autonomous driving. Traditional approaches rely on extensive annotated data for point cloud analysis, incurring high costs and time investments. In contrast, realworld image datasets offer abundant availability and substantial scale. To mitigate the burden of annotating 3D LiDAR point clouds, we propose two crossmodal knowledge distillation methods: Unsupervised Domain Adaptation Knowledge Distillation (UDAKD) and Feature and Semantic-based Knowledge Distillation (FSKD). Leveraging readily available spatio-temporally synchronized data from cameras and LiDARs in autonomous driving scenarios, we directly apply a pretrained 2D image model to unlabeled 2D data. Through crossmodal knowledge distillation with known 2D-3D correspondence, we actively align the output of the 3D network with the corresponding points of the 2D network, thereby obviating the necessity for 3D annotations. Our focus is on preserving modality-general information while filtering out modality-specific details during crossmodal distillation. To achieve this, we deploy self-calibrated convolution on 3D point clouds as the foundation of our domain adaptation module. Rigorous experimentation validates the effectiveness of our proposed methods, consistently surpassing the performance of state-of-the-art approaches in the field.

Related papers

• CMF-IoU: Multi-Stage Cross-Modal Fusion 3D Object Detection with IoU Joint Prediction [29.7092783661859]
  Multi-modal methods based on camera and LiDAR sensors have garnered significant attention in the field of 3D detection.<n>We introduce a multi-stage cross-modal fusion 3D detection framework, termed CMF-IOU, to address the challenge of aligning 3D spatial and 2D semantic information.
  arXiv  Detail & Related papers  (2025-08-18T13:32:07Z)
• xMOD: Cross-Modal Distillation for 2D/3D Multi-Object Discovery from 2D motion [4.878192303432336]
  DIOD-3D is the first baseline for multi-object discovery in 3D data using 2D motion.<n>xMOD is a cross-modal training framework that integrates 2D and 3D data while always using 2D motion cues.<n>Our approach yields a substantial performance improvement compared with the 2D object discovery state-of-the-art on all datasets.
  arXiv  Detail & Related papers  (2025-03-19T09:20:35Z)
• CleverDistiller: Simple and Spatially Consistent Cross-modal Distillation [7.246959698735599]
  CleverDistiller is a self-supervised, cross-modal 2D-to-3D KD framework.<n>It achieves state-of-the-art performance in both semantic segmentation and 3D object detection by up to 10% mIoU.
  arXiv  Detail & Related papers  (2025-03-12T22:18:29Z)
• LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving [52.83707400688378]
  LargeAD is a versatile and scalable framework designed for large-scale 3D pretraining across diverse real-world driving datasets.<n>Our framework leverages VFMs to extract semantically rich superpixels from 2D images, which are aligned with LiDAR point clouds to generate high-quality contrastive samples.<n>Our approach delivers significant performance improvements over state-of-the-art methods in both linear probing and fine-tuning tasks for both LiDAR-based segmentation and object detection.
  arXiv  Detail & Related papers  (2025-01-07T18:59:59Z)
• Cross-modal & Cross-domain Learning for Unsupervised LiDAR Semantic Segmentation [82.47872784972861]
  Cross-modal domain adaptation has been studied on the paired 2D image and 3D LiDAR data to ease the labeling costs for 3D LiDAR semantic segmentation (3DLSS) in the target domain. This paper studies a new 3DLSS setting where a 2D dataset with semantic annotations and a paired but unannotated 2D image and 3D LiDAR data (target) are available. To achieve 3DLSS in this scenario, we propose Cross-Modal and Cross-Domain Learning (CoMoDaL)
  arXiv  Detail & Related papers  (2023-08-05T14:00:05Z)
• 3D Point Cloud Pre-training with Knowledge Distillation from 2D Images [128.40422211090078]
  We propose a knowledge distillation method for 3D point cloud pre-trained models to acquire knowledge directly from the 2D representation learning model. Specifically, we introduce a cross-attention mechanism to extract concept features from 3D point cloud and compare them with the semantic information from 2D images. In this scheme, the point cloud pre-trained models learn directly from rich information contained in 2D teacher models.
  arXiv  Detail & Related papers  (2022-12-17T23:21:04Z)
• LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Driving [34.119642131912485]
  We present a more artful framework, LiDAR-guided Weakly Supervised Instance (LWSIS) LWSIS uses the off-the-shelf 3D data, i.e., Point Cloud, together with the 3D boxes, as natural weak supervisions for training the 2D image instance segmentation models. Our LWSIS not only exploits the complementary information in multimodal data during training, but also significantly reduces the cost of the dense 2D masks.
  arXiv  Detail & Related papers  (2022-12-07T08:08:01Z)
• Homography Loss for Monocular 3D Object Detection [54.04870007473932]
  A differentiable loss function, termed as Homography Loss, is proposed to achieve the goal, which exploits both 2D and 3D information. Our method yields the best performance compared with the other state-of-the-arts by a large margin on KITTI 3D datasets.
  arXiv  Detail & Related papers  (2022-04-02T03:48:03Z)
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception [122.53774221136193]
  State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution. A natural remedy is to utilize the 3D voxelization and 3D convolution network. We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
  arXiv  Detail & Related papers  (2021-09-12T06:25:11Z)
• Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation [87.54570024320354]
  State-of-the-art methods for large-scale driving-scene LiDAR semantic segmentation often project and process the point clouds in the 2D space. A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space. We develop a 3D cylinder partition and a 3D cylinder convolution based framework, termed as Cylinder3D, which exploits the 3D topology relations and structures of driving-scene point clouds.
  arXiv  Detail & Related papers  (2020-08-04T13:56:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.