Selective Transfer Learning of Cross-Modality Distillation for Monocular 3D Object Detection

• URL: http://arxiv.org/abs/2603.07464v1
• Date: Sun, 08 Mar 2026 05:05:07 GMT
• Title: Selective Transfer Learning of Cross-Modality Distillation for Monocular 3D Object Detection
• Abstract summary: Cross-modality knowledge distillation could effectively transfer depth information from LiDAR to image-based network.<n>However, modality gap between image and LiDAR seriously limits its accuracy.<n>We propose a selective learning approach named MonoSTL to overcome these issues.
• Score: 41.41435217865119
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monocular 3D object detection is a promising yet ill-posed task for autonomous vehicles due to the lack of accurate depth information. Cross-modality knowledge distillation could effectively transfer depth information from LiDAR to image-based network. However, modality gap between image and LiDAR seriously limits its accuracy. In this paper, we systematically investigate the negative transfer problem induced by modality gap in cross-modality distillation for the first time, including not only the architecture inconsistency issue but more importantly the feature overfitting issue. We propose a selective learning approach named MonoSTL to overcome these issues, which encourages positive transfer of depth information from LiDAR while alleviates the negative transfer on image-based network. On the one hand, we utilize similar architectures to ensure spatial alignment of features between image-based and LiDAR-based networks. On the other hand, we develop two novel distillation modules, namely Depth-Aware Selective Feature Distillation (DASFD) and Depth-Aware Selective Relation Distillation (DASRD), which selectively learn positive features and relationships of objects by integrating depth uncertainty into feature and relation distillations, respectively. Our approach can be seamlessly integrated into various CNN-based and DETR-based models, where we take three recent models on KITTI and a recent model on NuScenes for validation. Extensive experiments show that our approach considerably improves the accuracy of the base models and thereby achieves the best accuracy compared with all recently released SOTA models.

Related papers

• Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation [3.6903284283843942]
  Traditional approaches rely on extensive data for annotated point cloud analysis.<n>Realworld image datasets offer abundant availability and substantial scale.<n>We propose two crossmodal knowledge distillation methods: Unsupervised Domain Knowledge Distillation (UDAKD) and Feature and Semantic-based Knowledge Distillation (FSKD)
  arXiv  Detail & Related papers  (2025-08-30T06:34:39Z)
• Enhancing Steering Estimation with Semantic-Aware GNNs [41.89219383258699]
  hybrid architectures combine 3D neural network models with recurrent neural networks (RNNs) for temporal modeling.<n>We evaluate four hybrid 3D models, all of which outperform the 2D-only baseline.<n>We validate our approach on the KITTI dataset, achieving a 71% improvement over 2D-only models.
  arXiv  Detail & Related papers  (2025-03-21T13:58:08Z)
• 4D Contrastive Superflows are Dense 3D Representation Learners [62.433137130087445]
  We introduce SuperFlow, a novel framework designed to harness consecutive LiDAR-camera pairs for establishing pretraining objectives. To further boost learning efficiency, we incorporate a plug-and-play view consistency module that enhances alignment of the knowledge distilled from camera views.
  arXiv  Detail & Related papers  (2024-07-08T17:59:54Z)
• MonoSKD: General Distillation Framework for Monocular 3D Object Detection via Spearman Correlation Coefficient [11.48914285491747]
  Existing monocular 3D detection knowledge distillation methods usually project the LiDAR onto the image plane and train the teacher network accordingly. We propose MonoSKD, a novel Knowledge Distillation framework for Monocular 3D detection based on Spearman correlation coefficient. Our framework achieves state-of-the-art performance until submission with no additional inference computational cost.
  arXiv  Detail & Related papers  (2023-10-17T14:48:02Z)
• SimDistill: Simulated Multi-modal Distillation for BEV 3D Object Detection [56.24700754048067]
  Multi-view camera-based 3D object detection has become popular due to its low cost, but accurately inferring 3D geometry solely from camera data remains challenging. We propose a Simulated multi-modal Distillation (SimDistill) method by carefully crafting the model architecture and distillation strategy. Our SimDistill can learn better feature representations for 3D object detection while maintaining a cost-effective camera-only deployment.
  arXiv  Detail & Related papers  (2023-03-29T16:08:59Z)
• Boosting 3D Object Detection by Simulating Multimodality on Point Clouds [51.87740119160152]
  This paper presents a new approach to boost a single-modality (LiDAR) 3D object detector by teaching it to simulate features and responses that follow a multi-modality (LiDAR-image) detector. The approach needs LiDAR-image data only when training the single-modality detector, and once well-trained, it only needs LiDAR data at inference. Experimental results on the nuScenes dataset show that our approach outperforms all SOTA LiDAR-only 3D detectors.
  arXiv  Detail & Related papers  (2022-06-30T01:44:30Z)
• MonoDistill: Learning Spatial Features for Monocular 3D Object Detection [80.74622486604886]
  We propose a simple and effective scheme to introduce the spatial information from LiDAR signals to the monocular 3D detectors. We use the resulting data to train a 3D detector with the same architecture as the baseline model. Experimental results show that the proposed method can significantly boost the performance of the baseline model.
  arXiv  Detail & Related papers  (2022-01-26T09:21:41Z)
• SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks [81.64530401885476]
  We propose a self-supervised LiDAR odometry method, dubbed SelfVoxeLO, to tackle these two difficulties. Specifically, we propose a 3D convolution network to process the raw LiDAR data directly, which extracts features that better encode the 3D geometric patterns. We evaluate our method's performances on two large-scale datasets, i.e., KITTI and Apollo-SouthBay.
  arXiv  Detail & Related papers  (2020-10-19T09:23:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.