Novel Class Discovery for 3D Point Cloud Semantic Segmentation
- URL: http://arxiv.org/abs/2303.11610v1
- Date: Tue, 21 Mar 2023 06:10:39 GMT
- Title: Novel Class Discovery for 3D Point Cloud Semantic Segmentation
- Abstract summary: Novel class discovery (NCD) for semantic segmentation is the task of learning a model that can segment unlabelled (novel) classes using only the supervision from labelled (base) classes.
This paper is presented to advance the state of the art on point cloud data analysis in four directions.
- Score: 13.024681120553906
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Novel class discovery (NCD) for semantic segmentation is the task of learning
a model that can segment unlabelled (novel) classes using only the supervision
from labelled (base) classes. This problem has recently been pioneered for 2D
image data, but no work exists for 3D point cloud data. In fact, the
assumptions made for 2D are loosely applicable to 3D in this case. This paper
is presented to advance the state of the art on point cloud data analysis in
four directions. Firstly, we address the new problem of NCD for point cloud
semantic segmentation. Secondly, we show that the transposition of the only
existing NCD method for 2D semantic segmentation to 3D data is suboptimal.
Thirdly, we present a new method for NCD based on online clustering that
exploits uncertainty quantification to produce prototypes for pseudo-labelling
the points of the novel classes. Lastly, we introduce a new evaluation protocol
to assess the performance of NCD for point cloud semantic segmentation. We
thoroughly evaluate our method on SemanticKITTI and SemanticPOSS datasets,
showing that it can significantly outperform the baseline. Project page at this
link: https://github.com/LuigiRiz/NOPS.
Related papers
- Bayesian Self-Training for Semi-Supervised 3D Segmentation [59.544558398992386]
3D segmentation is a core problem in computer vision.
densely labeling 3D point clouds to employ fully-supervised training remains too labor intensive and expensive.
Semi-supervised training provides a more practical alternative, where only a small set of labeled data is given, accompanied by a larger unlabeled set.
arXiv Detail & Related papers (2024-09-12T14:54:31Z) - Multi-modality Affinity Inference for Weakly Supervised 3D Semantic
Segmentation [47.81638388980828]
We propose a simple yet effective scene-level weakly supervised point cloud segmentation method with a newly introduced multi-modality point affinity inference module.
Our method outperforms the state-of-the-art by 4% to 6% mIoU on the ScanNet and S3DIS benchmarks.
arXiv Detail & Related papers (2023-12-27T14:01:35Z) - Novel class discovery meets foundation models for 3D semantic segmentation [21.879254503403516]
This paper introduces the novel task of NCD for point cloud semantic segmentation.
It demonstrates that directly transposing the only existing NCD method for 2D semantic segmentation to 3D data yields suboptimal results.
A new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented.
arXiv Detail & Related papers (2023-12-06T09:59:30Z) - Prototype Adaption and Projection for Few- and Zero-shot 3D Point Cloud
Semantic Segmentation [30.18333233940194]
We address the challenging task of few-shot and zero-shot 3D point cloud semantic segmentation.
Our proposed method surpasses state-of-the-art algorithms by a considerable 7.90% and 14.82% under the 2-way 1-shot setting on S3DIS and ScanNet benchmarks, respectively.
arXiv Detail & Related papers (2023-05-23T17:58:05Z) - Dual Adaptive Transformations for Weakly Supervised Point Cloud
Segmentation [78.6612285236938]
We propose a novel DAT (textbfDual textbfAdaptive textbfTransformations) model for weakly supervised point cloud segmentation.
We evaluate our proposed DAT model with two popular backbones on the large-scale S3DIS and ScanNet-V2 datasets.
arXiv Detail & Related papers (2022-07-19T05:43:14Z) - PointAttN: You Only Need Attention for Point Cloud Completion [89.88766317412052]
Point cloud completion refers to completing 3D shapes from partial 3D point clouds.
We propose a novel neural network for processing point cloud in a per-point manner to eliminate kNNs.
The proposed framework, namely PointAttN, is simple, neat and effective, which can precisely capture the structural information of 3D shapes.
arXiv Detail & Related papers (2022-03-16T09:20:01Z) - Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based
Perception [122.53774221136193]
State-of-the-art methods for driving-scene LiDAR-based perception often project the point clouds to 2D space and then process them via 2D convolution.
A natural remedy is to utilize the 3D voxelization and 3D convolution network.
We propose a new framework for the outdoor LiDAR segmentation, where cylindrical partition and asymmetrical 3D convolution networks are designed to explore the 3D geometric pattern.
arXiv Detail & Related papers (2021-09-12T06:25:11Z) - Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method.
Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
arXiv Detail & Related papers (2020-06-22T08:05:25Z) - Label-Efficient Learning on Point Clouds using Approximate Convex
Decompositions [43.1279121348315]
We investigate the use of Approximate Convex Decompositions (ACD) as a self-supervisory signal for label-efficient learning of point cloud representations.
We show that using ACD to approximate ground truth segmentation provides excellent self-supervision for learning 3D point cloud representations.
arXiv Detail & Related papers (2020-03-30T21:44:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.