Sketchy Bounding-box Supervision for 3D Instance Segmentation

• URL: http://arxiv.org/abs/2505.16399v1
• Date: Thu, 22 May 2025 08:49:49 GMT
• Title: Sketchy Bounding-box Supervision for 3D Instance Segmentation
• Authors: Qian Deng, Le Hui, Jin Xie, Jian Yang,
• Abstract summary: We propose Sketchy-3DIS, a novel weakly supervised 3D instance segmentation framework.<n>We first propose an adaptive box-to-point pseudo labeler that adaptively learns to assign points located in the overlapped parts between two sketchy bounding boxes to the correct instance.<n>We then present a coarse-to-fine instance segmentator that first predicts coarse instances from the entire point cloud and then learns fine instances based on the region of coarse instances.
• Score: 27.26709842992742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bounding box supervision has gained considerable attention in weakly supervised 3D instance segmentation. While this approach alleviates the need for extensive point-level annotations, obtaining accurate bounding boxes in practical applications remains challenging. To this end, we explore the inaccurate bounding box, named sketchy bounding box, which is imitated through perturbing ground truth bounding box by adding scaling, translation, and rotation. In this paper, we propose Sketchy-3DIS, a novel weakly 3D instance segmentation framework, which jointly learns pseudo labeler and segmentator to improve the performance under the sketchy bounding-box supervisions. Specifically, we first propose an adaptive box-to-point pseudo labeler that adaptively learns to assign points located in the overlapped parts between two sketchy bounding boxes to the correct instance, resulting in compact and pure pseudo instance labels. Then, we present a coarse-to-fine instance segmentator that first predicts coarse instances from the entire point cloud and then learns fine instances based on the region of coarse instances. Finally, by using the pseudo instance labels to supervise the instance segmentator, we can gradually generate high-quality instances through joint training. Extensive experiments show that our method achieves state-of-the-art performance on both the ScanNetV2 and S3DIS benchmarks, and even outperforms several fully supervised methods using sketchy bounding boxes. Code is available at https://github.com/dengq7/Sketchy-3DIS.

Related papers

• Bayesian Self-Training for Semi-Supervised 3D Segmentation [59.544558398992386]
  3D segmentation is a core problem in computer vision. densely labeling 3D point clouds to employ fully-supervised training remains too labor intensive and expensive. Semi-supervised training provides a more practical alternative, where only a small set of labeled data is given, accompanied by a larger unlabeled set.
  arXiv  Detail & Related papers  (2024-09-12T14:54:31Z)
• Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation [50.51125319374404]
  We propose a novel self-training network InsTeacher3D to explore and exploit pure instance knowledge from unlabeled data. Experimental results on multiple large-scale datasets show that the InsTeacher3D significantly outperforms prior state-of-the-art semi-supervised approaches.
  arXiv  Detail & Related papers  (2024-06-24T16:35:58Z)
• When 3D Bounding-Box Meets SAM: Point Cloud Instance Segmentation with Weak-and-Noisy Supervision [20.625754683390536]
  We propose a complementary image prompt-induced weakly-supervised point cloud instance segmentation (CIP-WPIS) method. We leverage pretrained knowledge embedded in the 2D foundation model SAM and 3D geometric prior to achieve accurate point-wise instance labels. Our method is robust against noisy 3D bounding-box annotations and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-09-02T05:17:03Z)
• Weakly Supervised 3D Instance Segmentation without Instance-level Annotations [57.615325809883636]
  3D semantic scene understanding tasks have achieved great success with the emergence of deep learning, but often require a huge amount of manually annotated training data. We propose the first weakly-supervised 3D instance segmentation method that only requires categorical semantic labels as supervision. By generating pseudo instance labels from categorical semantic labels, our designed approach can also assist existing methods for learning 3D instance segmentation at reduced annotation cost.
  arXiv  Detail & Related papers  (2023-08-03T12:30:52Z)
• GaPro: Box-Supervised 3D Point Cloud Instance Segmentation Using Gaussian Processes as Pseudo Labelers [14.88505076974645]
  GaPro is a new instance segmentation for 3D point clouds using axis-aligned 3D bounding box supervision. Our two-step approach involves generating pseudo labels from box annotations and training a 3DIS network with the resulting labels. Our experiments show that GaPro outperforms previous weakly supervised 3D instance segmentation methods.
  arXiv  Detail & Related papers  (2023-07-25T04:43:22Z)
• Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation [10.968271388503986]
  We propose a 3D instance segmentation framework that can achieve good performance by annotating only one point for each instance. Our method achieves state-of-the-art performance in the weakly supervised point cloud instance segmentation task, and even outperforms some fully supervised methods.
  arXiv  Detail & Related papers  (2022-10-11T15:22:22Z)
• Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision [63.429704654271475]
  We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
  arXiv  Detail & Related papers  (2022-08-10T02:14:39Z)
• Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes [38.60444957213202]
  We look at weakly-supervised 3D semantic instance segmentation. Key idea is to leverage 3D bounding box labels which are easier and faster to annotate. We show that it is possible to train dense segmentation models using only bounding box labels.
  arXiv  Detail & Related papers  (2022-06-02T17:59:57Z)
• Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision [65.19589997822155]
  We introduce a neural architecture, termed Box2Seg, to learn point-level semantics of 3D point clouds with bounding box-level supervision. We show that the proposed network can be trained with cheap, or even off-the-shelf bounding box-level annotations and subcloud-level tags.
  arXiv  Detail & Related papers  (2022-01-09T09:07:48Z)
• Dense Supervision Propagation for Weakly Supervised Semantic Segmentation on 3D Point Clouds [59.63231842439687]
  We train a semantic point cloud segmentation network with only a small portion of points being labeled. We propose a cross-sample feature reallocating module to transfer similar features and therefore re-route the gradients across two samples. Our weakly supervised method with only 10% and 1% of labels can produce compatible results with the fully supervised counterpart.
  arXiv  Detail & Related papers  (2021-07-23T14:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.