MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection

• URL: http://arxiv.org/abs/2404.04910v1
• Date: Sun, 7 Apr 2024 10:39:04 GMT
• Title: MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection
• Authors: Hou-I Liu, Christine Wu, Jen-Hao Cheng, Wenhao Chai, Shian-Yun Wang, Gaowen Liu, Jenq-Neng Hwang, Hong-Han Shuai, Wen-Huang Cheng,
• Abstract summary: Monocular 3D object detection is an indispensable research topic in autonomous driving. The challenges of Mono3D lie in understanding 3D scene geometry and reconstructing 3D object information from a single image. Previous methods attempted to transfer 3D information directly from the LiDAR-based teacher to the camera-based student.
• Score: 42.4932760909941
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monocular 3D object detection (Mono3D) is an indispensable research topic in autonomous driving, thanks to the cost-effective monocular camera sensors and its wide range of applications. Since the image perspective has depth ambiguity, the challenges of Mono3D lie in understanding 3D scene geometry and reconstructing 3D object information from a single image. Previous methods attempted to transfer 3D information directly from the LiDAR-based teacher to the camera-based student. However, a considerable gap in feature representation makes direct cross-modal distillation inefficient, resulting in a significant performance deterioration between the LiDAR-based teacher and the camera-based student. To address this issue, we propose the Teaching Assistant Knowledge Distillation (MonoTAKD) to break down the learning objective by integrating intra-modal distillation with cross-modal residual distillation. In particular, we employ a strong camera-based teaching assistant model to distill powerful visual knowledge effectively through intra-modal distillation. Subsequently, we introduce the cross-modal residual distillation to transfer the 3D spatial cues. By acquiring both visual knowledge and 3D spatial cues, the predictions of our approach are rigorously evaluated on the KITTI 3D object detection benchmark and achieve state-of-the-art performance in Mono3D.

Related papers

• Weakly Supervised Monocular 3D Detection with a Single-View Image [58.57978772009438]
  Monocular 3D detection aims for precise 3D object localization from a single-view image. We propose SKD-WM3D, a weakly supervised monocular 3D detection framework. We show that SKD-WM3D surpasses the state-of-the-art clearly and is even on par with many fully supervised methods.
  arXiv  Detail & Related papers  (2024-02-29T13:26:47Z)
• ODM3D: Alleviating Foreground Sparsity for Semi-Supervised Monocular 3D Object Detection [15.204935788297226]
  ODM3D framework entails cross-modal knowledge distillation at various levels to inject LiDAR-domain knowledge into a monocular detector during training. By identifying foreground sparsity as the main culprit behind existing methods' suboptimal training, we exploit the precise localisation information embedded in LiDAR points. Our method ranks 1st in both KITTI validation and test benchmarks, significantly surpassing all existing monocular methods, supervised or semi-supervised.
  arXiv  Detail & Related papers  (2023-10-28T07:12:09Z)
• SimDistill: Simulated Multi-modal Distillation for BEV 3D Object Detection [56.24700754048067]
  Multi-view camera-based 3D object detection has become popular due to its low cost, but accurately inferring 3D geometry solely from camera data remains challenging. We propose a Simulated multi-modal Distillation (SimDistill) method by carefully crafting the model architecture and distillation strategy. Our SimDistill can learn better feature representations for 3D object detection while maintaining a cost-effective camera-only deployment.
  arXiv  Detail & Related papers  (2023-03-29T16:08:59Z)
• X$^3$KD: Knowledge Distillation Across Modalities, Tasks and Stages for Multi-Camera 3D Object Detection [45.32989526953387]
  This paper introduces X$3$KD, a comprehensive knowledge distillation framework across different modalities, tasks, and stages for multi-camera 3DOD. After the transformation, we apply cross-modal feature distillation (X-FD) and adversarial training (X-AT) to improve the 3D world representation of multi-camera features. Our final X$3$KD model outperforms previous state-of-the-art approaches on the nuScenes and datasets.
  arXiv  Detail & Related papers  (2023-03-03T20:29:49Z)
• Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection [10.84784828447741]
  ADD is an Attention-based Depth knowledge Distillation framework with 3D-aware positional encoding. Credit to our teacher design, our framework is seamless, domain-gap free, easily implementable, and is compatible with object-wise ground-truth depth. We implement our framework on three representative monocular detectors, and we achieve state-of-the-art performance with no additional inference computational cost.
  arXiv  Detail & Related papers  (2022-11-30T06:39:25Z)
• MonoDistill: Learning Spatial Features for Monocular 3D Object Detection [80.74622486604886]
  We propose a simple and effective scheme to introduce the spatial information from LiDAR signals to the monocular 3D detectors. We use the resulting data to train a 3D detector with the same architecture as the baseline model. Experimental results show that the proposed method can significantly boost the performance of the baseline model.
  arXiv  Detail & Related papers  (2022-01-26T09:21:41Z)
• SGM3D: Stereo Guided Monocular 3D Object Detection [62.11858392862551]
  We propose a stereo-guided monocular 3D object detection network, termed SGM3D. We exploit robust 3D features extracted from stereo images to enhance the features learned from the monocular image. Our method can be integrated into many other monocular approaches to boost performance without introducing any extra computational cost.
  arXiv  Detail & Related papers  (2021-12-03T13:57:14Z)
• Kinematic 3D Object Detection in Monocular Video [123.7119180923524]
  We propose a novel method for monocular video-based 3D object detection which carefully leverages kinematic motion to improve precision of 3D localization. We achieve state-of-the-art performance on monocular 3D object detection and the Bird's Eye View tasks within the KITTI self-driving dataset.
  arXiv  Detail & Related papers  (2020-07-19T01:15:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.