GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure

• URL: http://arxiv.org/abs/2608.14428v1
• Date: Fri, 14 Aug 2026 16:10:35 GMT
• Title: GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure
• Abstract summary: 3D object detection from LiDAR point clouds is a core problem in autonomous driving.<n>Recent advances in self-supervised learning (SSL) enable scalable pretraining and transfers well to per-point tasks.<n>We analyze recent SSL methods and find that most objectives are defined only on measured LiDAR returns from visible surfaces.<n>We propose GhostPoint, an SSL framework that hallucinates latent features in local neighborhoods around discovered instances.
• Score: 58.617405464262646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D object detection from LiDAR point clouds is a core problem in autonomous driving. Recent advances in self-supervised learning (SSL) enable scalable pretraining and transfers well to per-point tasks such as semantic and panoptic segmentation, but transfer to 3D detection remains weaker. We analyze recent SSL methods and find that most objectives are defined only on measured LiDAR returns from visible surfaces, leaving occluded and unobserved regions unconstrained. This visible-surface bias can be sufficient for point-wise prediction, but 3D detection requires robustness to missing structure. To address this gap, we propose GhostPoint, an SSL framework that hallucinates latent features in local neighborhoods around discovered instances, generated via a novel instance voxel dilation. In GhostPoint, an encoder processes observed returns, and an additional predictor infers neighborhood representations from observed context. In addition to standard encoder-level supervision, we introduce a predictor-level supervision scheme on sampled voxels from generated neighborhoods. Specifically, observed (visible/masked) voxels match teacher-encoder targets, while unobserved voxels match teacher-predictor hallucinations. This design encourages the learned representation to explicitly model structure beyond observed returns. Extensive evaluations on nuScenes and Waymo demonstrate that our method achieves state-of-the-art performance, consistently improving downstream 3D detection, especially under sparse scans and limited labels.

Related papers

• UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations [61.85916280854257]
  Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input.<n>We introduce textbfUniProbe, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass.
  arXiv  Detail & Related papers  (2026-08-11T12:01:59Z)
• Explainability-Aware Frustum Attack: Exposing Structural Vulnerabilities in LiDAR-Based 3D Object Detectors [5.47472857635307]
  Saliency-LiDAR method produces universal saliency maps for LiDAR-based 3D object detectors.<n>We design the Explainability-aware Frustum Attack (EFA), which selectively perturbs only the most influential frustums rather than uniformly attacking entire object regions.<n>EFA reduces detection recall by more than 15 percentage points while requiring 25-50% fewer frustums than the state-of-the-art non-saliency-aware baseline.
  arXiv  Detail & Related papers  (2026-06-29T08:40:16Z)
• On the Feasibility and Opportunity of Autoregressive 3D Object Detection [60.86546723351944]
  AutoReg3D is an autoregressive 3D detector that casts detection as sequence generation.<n>It is compatible across diverse point-cloud or backbones and attains competitive nuScenes performance without anchors or NMS.
  arXiv  Detail & Related papers  (2026-03-09T05:46:53Z)
• DOS: Distilling Observable Softmaps of Zipfian Prototypes for Self-Supervised Point Representation [25.293422897925698]
  DOS (Distilling Observable Softmaps) is a novel SSL framework that self-distills semantic relevance softmaps only at observable points.<n> DOS outperforms current state-of-the-art methods on semantic segmentation and 3D object detection.<n>Our results demonstrate that observable-point softmaps distillation offers a scalable and effective paradigm for learning robust 3D representations.
  arXiv  Detail & Related papers  (2025-12-12T11:07:40Z)
• Hierarchical Supervision and Shuffle Data Augmentation for 3D Semi-Supervised Object Detection [90.32180043449263]
  State-of-the-art 3D object detectors are usually trained on large-scale datasets with high-quality 3D annotations. A natural remedy is to adopt semi-supervised learning (SSL) by leveraging a limited amount of labeled samples and abundant unlabeled samples. This paper introduces a novel approach of Hierarchical Supervision and Shuffle Data Augmentation (HSSDA), which is a simple yet effective teacher-student framework.
  arXiv  Detail & Related papers  (2023-04-04T02:09:32Z)
• MAELi: Masked Autoencoder for Large-Scale LiDAR Point Clouds [13.426810473131642]
  Masked AutoEncoder for LiDAR point clouds (MAELi) intuitively leverages the sparsity of LiDAR point clouds in both the encoder and decoder during reconstruction. In a novel reconstruction approach, MAELi distinguishes between empty and occluded space. Thereby, without any ground truth whatsoever and trained on single frames only, MAELi obtains an understanding of the underlying 3D scene geometry and semantics.
  arXiv  Detail & Related papers  (2022-12-14T13:10:27Z)
• UpCycling: Semi-supervised 3D Object Detection without Sharing Raw-level Unlabeled Scenes [7.32610370107512]
  UpCycling is a novel SSL framework for 3D object detection with zero additional raw-level point cloud. We introduce hybrid pseudo labels, feature-level Ground Truth sampling (F-GT) and Rotation (F-RoT) UpCycling significantly outperforms the state-of-the-art SSL methods that utilize raw-point scenes.
  arXiv  Detail & Related papers  (2022-11-22T02:04:09Z)
• Self-Supervised Point Cloud Representation Learning with Occlusion Auto-Encoder [63.77257588569852]
  We present 3D Occlusion Auto-Encoder (3D-OAE) for learning representations for point clouds. Our key idea is to randomly occlude some local patches of the input point cloud and establish the supervision via recovering the occluded patches. In contrast with previous methods, our 3D-OAE can remove a large proportion of patches and predict them only with a small number of visible patches.
  arXiv  Detail & Related papers  (2022-03-26T14:06:29Z)
• SESS: Self-Ensembling Semi-Supervised 3D Object Detection [138.80825169240302]
  We propose SESS, a self-ensembling semi-supervised 3D object detection framework. Specifically, we design a thorough perturbation scheme to enhance generalization of the network on unlabeled and new unseen data. Our SESS achieves competitive performance compared to the state-of-the-art fully-supervised method by using only 50% labeled data.
  arXiv  Detail & Related papers  (2019-12-26T08:48:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.