SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection

• URL: http://arxiv.org/abs/2604.18476v1
• Date: Mon, 20 Apr 2026 16:28:01 GMT
• Title: SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection
• Abstract summary: Camera-only 3D object detection has emerged as a cost-effective and scalable alternative to LiDAR for autonomous driving.<n>In practice, many rare but safety-critical categories such as children, strollers, or emergency vehicles are heavily underrepresented.<n>We introduce SemLT3D, a Semantic-Guided Expert Distillation framework designed to enrich the representation space for underrepresented classes.
• Score: 14.96234618252065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Camera-only 3D object detection has emerged as a cost-effective and scalable alternative to LiDAR for autonomous driving, yet existing methods primarily prioritize overall performance while overlooking the severe long-tail imbalance inherent in real-world datasets. In practice, many rare but safety-critical categories such as children, strollers, or emergency vehicles are heavily underrepresented, leading to biased learning and degraded performance. This challenge is further exacerbated by pronounced inter-class ambiguity (e.g., visually similar subclasses) and substantial intra-class diversity (e.g., objects varying widely in appearance, scale, pose, or context), which together hinder reliable long-tail recognition. In this work, we introduce SemLT3D, a Semantic-Guided Expert Distillation framework designed to enrich the representation space for underrepresented classes through semantic priors. SemLT3D consists of: (1) a language-guided mixture-of-experts module that routes 3D queries to specialized experts according to their semantic affinity, enabling the model to better disentangle confusing classes and specialize on tail distributions; and (2) a semantic projection distillation pipeline that aligns 3D queries with CLIP-informed 2D semantics, producing more coherent and discriminative features across diverse visual manifestations. Although motivated by long-tail imbalance, the semantically structured learning in SemLT3D also improves robustness under broader appearance variations and challenging corner cases, offering a principled step toward more reliable camera-only 3D perception.

Related papers

• VLRC: Vision-Language Reprojection Consistency as a scalable signal for better feed-forward 3D pretraining [8.146936489399293]
  Vision-Language Reprojection Consistency (VLRC) exploits frozen vision-language representations as semantic multi-view supervision.<n>VLRC integrates seamlessly with both self-supervised monocular reconstruction and supervised-pretrained feed-forward 3D models.<n>Experiments on indoor and outdoor benchmarks demonstrate consistent gains in 3D reconstruction accuracy and zero-shot open-vocabulary 3D semantic segmentation.
  arXiv  Detail & Related papers  (2026-07-02T18:49:34Z)
• Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge [45.19482892758984]
  Affordance segmentation aims to parse 3D objects into functionally distinct parts, bridging recognition and interaction for applications in robotic manipulation, embodied AI, and AR.<n>We introduce Cross-Modal Affinity Transfer (CMAT), a pre-training strategy that aligns a 3D encoder with lifted 2D semantics and jointly optimize reconstruction, affinity, and diversity to yield semantically organized representations.<n>We further design the Cross-modal Affordance Transformer (CAST), which integrates multi-modal prompts with CMAT-pretrained features to generate precise, prompt-aware segmentation maps.
  arXiv  Detail & Related papers  (2025-10-09T15:01:26Z)
• Semantic Causality-Aware Vision-Based 3D Occupancy Prediction [63.752869043357585]
  Vision-based 3D semantic occupancy prediction is a critical task in 3D vision.<n>Existing methods, however, often rely on modular pipelines.<n>We propose a novel causal loss that enables holistic, end-to-end supervision of the modular 2D-to-3D transformation pipeline.
  arXiv  Detail & Related papers  (2025-09-10T08:29:22Z)
• CleverDistiller: Simple and Spatially Consistent Cross-modal Distillation [7.246959698735599]
  CleverDistiller is a self-supervised, cross-modal 2D-to-3D KD framework.<n>It achieves state-of-the-art performance in both semantic segmentation and 3D object detection by up to 10% mIoU.
  arXiv  Detail & Related papers  (2025-03-12T22:18:29Z)
• CA-W3D: Leveraging Context-Aware Knowledge for Weakly Supervised Monocular 3D Detection [5.881158575425763]
  We propose a Context-Aware Weak Supervision for Monocular 3D object detection, namely CA-W3D, to address this limitation in a two-stage training paradigm.<n>Specifically, we first introduce a pre-training stage employing Region-wise Object Contrastive Matching (ROCM), which aligns regional object embeddings derived from a trainable monocular 3D encoder and a frozen open-vocabulary 2D visual grounding model.<n>In the second stage, we incorporate a pseudo-label training process with a Dual-to-One Distillation (D2OD) mechanism, which effectively transfers contextual priors into
  arXiv  Detail & Related papers  (2025-03-06T07:02:13Z)
• GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistency [50.11520458252128]
  Existing 3D affordance learning methods struggle with generalization and robustness due to limited annotated data.<n>We propose GEAL, a novel framework designed to enhance the generalization and robustness of 3D affordance learning by leveraging large-scale pre-trained 2D models.<n>GEAL consistently outperforms existing methods across seen and novel object categories, as well as corrupted data.
  arXiv  Detail & Related papers  (2024-12-12T17:59:03Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• 3D-LFM: Lifting Foundation Model [29.48835001900286]
  deep learning has expanded our capability to reconstruct a wide range of object classes. Our approach harnesses the inherent permutation equivariance transformers to manage varying number points per 3D data instance. We demonstrate state the art performance across 2D-3D lifting task benchmarks.
  arXiv  Detail & Related papers  (2023-12-19T06:38:18Z)
• Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection [66.74183705987276]
  We introduce a framework to improve the camera-only apprentice model, including an apprentice-friendly multi-modal expert and temporal-fusion-friendly distillation supervision. With those improvements, our camera-only apprentice VCD-A sets new state-of-the-art on nuScenes with a score of 63.1% NDS.
  arXiv  Detail & Related papers  (2023-10-24T09:29:26Z)
• Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds [96.9027094562957]
  We introduce a-temporal representation learning framework, capable of learning from unlabeled tasks. Inspired by how infants learn from visual data in the wild, we explore rich cues derived from the 3D data. STRL takes two temporally-related frames from a 3D point cloud sequence as the input, transforms it with the spatial data augmentation, and learns the invariant representation self-supervisedly.
  arXiv  Detail & Related papers  (2021-09-01T04:17:11Z)
• Unsupervised Cross-Modal Alignment for Multi-Person 3D Pose Estimation [52.94078950641959]
  We present a deployment friendly, fast bottom-up framework for multi-person 3D human pose estimation. We adopt a novel neural representation of multi-person 3D pose which unifies the position of person instances with their corresponding 3D pose representation. We propose a practical deployment paradigm where paired 2D or 3D pose annotations are unavailable.
  arXiv  Detail & Related papers  (2020-08-04T07:54:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.