Seg2Box: 3D Object Detection by Point-Wise Semantics Supervision

• URL: http://arxiv.org/abs/2503.16811v1
• Date: Fri, 21 Mar 2025 02:39:32 GMT
• Title: Seg2Box: 3D Object Detection by Point-Wise Semantics Supervision
• Authors: Maoji Zheng, Ziyu Xu, Qiming Xia, Hai Wu, Chenglu Wen, Cheng Wang,
• Abstract summary: LiDAR-based 3D object detection and semantic segmentation are critical tasks in 3D scene understanding.<n>Traditional detection and methods supervise their models through bounding box labels and semantic mask labels.<n>This paper aims to eliminate the redundancy by supervising 3D object detection using only semantic labels.
• Score: 15.996707255179668
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LiDAR-based 3D object detection and semantic segmentation are critical tasks in 3D scene understanding. Traditional detection and segmentation methods supervise their models through bounding box labels and semantic mask labels. However, these two independent labels inherently contain significant redundancy. This paper aims to eliminate the redundancy by supervising 3D object detection using only semantic labels. However, the challenge arises due to the incomplete geometry structure and boundary ambiguity of point-cloud instances, leading to inaccurate pseudo labels and poor detection results. To address these challenges, we propose a novel method, named Seg2Box. We first introduce a Multi-Frame Multi-Scale Clustering (MFMS-C) module, which leverages the spatio-temporal consistency of point clouds to generate accurate box-level pseudo-labels. Additionally, the Semantic?Guiding Iterative-Mining Self-Training (SGIM-ST) module is proposed to enhance the performance by progressively refining the pseudo-labels and mining the instances without generating pseudo-labels. Experiments on the Waymo Open Dataset and nuScenes Dataset show that our method significantly outperforms other competitive methods by 23.7\% and 10.3\% in mAP, respectively. The results demonstrate the great label-efficient potential and advancement of our method.

Related papers

• SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts [13.349110509879312]
  sparsely-supervised 3D object detection has gained great attention, achieving performance close to fully-supervised 3D objectors. We propose a boosting strategy, termed SP3D, to boost the 3D detector with robust feature discrimination capability under sparse annotation settings. Experiments have validated that SP3D can enhance the performance of sparsely supervised detectors by a large margin under meager labeling conditions.
  arXiv  Detail & Related papers  (2025-03-09T06:08:04Z)
• Label-Efficient LiDAR Panoptic Segmentation [22.440065488051047]
  Limited-Label LiDAR Panoptic (L3PS)<n>We develop a label-efficient 2D network to generate panoptic pseudo-labels from annotated images.<n>We then introduce a novel 3D refinement module that capitalizes on the geometric properties of point clouds.
  arXiv  Detail & Related papers  (2025-03-04T07:58:15Z)
• Decoupled Pseudo-labeling for Semi-Supervised Monocular 3D Object Detection [108.672972439282]
  We introduce a novel decoupled pseudo-labeling (DPL) approach for SSM3OD. Our approach features a Decoupled Pseudo-label Generation (DPG) module, designed to efficiently generate pseudo-labels. We also present a DepthGradient Projection (DGP) module to mitigate optimization conflicts caused by noisy depth supervision of pseudo-labels.
  arXiv  Detail & Related papers  (2024-03-26T05:12:18Z)
• Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning [59.64695628433855]
  We propose a novel cross-modality weakly supervised method for 3D segmentation, incorporating complementary information from unlabeled images. Basically, we design a dual-branch network equipped with an active labeling strategy, to maximize the power of tiny parts of labels. Our method even outperforms the state-of-the-art fully supervised competitors with less than 1% actively selected annotations.
  arXiv  Detail & Related papers  (2022-09-16T07:59:04Z)
• Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision [63.429704654271475]
  We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
  arXiv  Detail & Related papers  (2022-08-10T02:14:39Z)
• Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision [65.19589997822155]
  We introduce a neural architecture, termed Box2Seg, to learn point-level semantics of 3D point clouds with bounding box-level supervision. We show that the proposed network can be trained with cheap, or even off-the-shelf bounding box-level annotations and subcloud-level tags.
  arXiv  Detail & Related papers  (2022-01-09T09:07:48Z)
• Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
  We train a semantic point cloud segmentation network with only a small portion of points being labeled. We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples. Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
  arXiv  Detail & Related papers  (2021-07-23T14:34:57Z)
• Unsupervised Object Detection with LiDAR Clues [70.73881791310495]
  We present the first practical method for unsupervised object detection with the aid of LiDAR clues. In our approach, candidate object segments based on 3D point clouds are firstly generated. Then, an iterative segment labeling process is conducted to assign segment labels and to train a segment labeling network. The labeling process is carefully designed so as to mitigate the issue of long-tailed and open-ended distribution.
  arXiv  Detail & Related papers  (2020-11-25T18:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.