Label Name is Mantra: Unifying Point Cloud Segmentation across
Heterogeneous Datasets
- URL: http://arxiv.org/abs/2303.10585v1
- Date: Sun, 19 Mar 2023 06:14:22 GMT
- Title: Label Name is Mantra: Unifying Point Cloud Segmentation across
Heterogeneous Datasets
- Authors: Yixun Liang, Hao He, Shishi Xiao, Hao Lu and Yingcong Chen
- Abstract summary: We propose a principled approach that supports learning from heterogeneous datasets with different label sets.
Our idea is to utilize a pre-trained language model to embed discrete labels to a continuous latent space with the help of their label names.
Our model outperforms the state-of-the-art by a large margin.
- Score: 17.503843467554592
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Point cloud segmentation is a fundamental task in 3D vision that serves a
wide range of applications. Although great progresses have been made these
years, its practical usability is still limited by the availability of training
data. Existing approaches cannot make full use of multiple datasets on hand due
to the label mismatch among different datasets. In this paper, we propose a
principled approach that supports learning from heterogeneous datasets with
different label sets. Our idea is to utilize a pre-trained language model to
embed discrete labels to a continuous latent space with the help of their label
names. This unifies all labels of different datasets, so that joint training is
doable. Meanwhile, classifying points in the continuous 3D space by their
vocabulary tokens significantly increase the generalization ability of the
model in comparison with existing approaches that have fixed decoder
architecture. Besides, we also integrate prompt learning in our framework to
alleviate data shifts among different data sources. Extensive experiments
demonstrate that our model outperforms the state-of-the-art by a large margin.
Related papers
- LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds.
Our method co-designs an efficient labeling process with semi/weakly supervised learning.
Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
arXiv Detail & Related papers (2022-10-14T19:13:36Z) - Teachers in concordance for pseudo-labeling of 3D sequential data [1.1610573589377013]
We propose to leverage sequences of point clouds to boost the pseudolabeling technique in a teacher-student setup via training multiple teachers.
This set of teachers, dubbed Concordance, provides higher quality pseudo-labels for student training than standard methods.
Our approach, which uses only 20% manual labels, outperforms some fully supervised methods.
arXiv Detail & Related papers (2022-07-13T09:40:22Z) - Learning Semantic Segmentation from Multiple Datasets with Label Shifts [101.24334184653355]
This paper proposes UniSeg, an effective approach to automatically train models across multiple datasets with differing label spaces.
Specifically, we propose two losses that account for conflicting and co-occurring labels to achieve better generalization performance in unseen domains.
arXiv Detail & Related papers (2022-02-28T18:55:19Z) - COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse
LiDAR datasets [3.8243923744440926]
Transfer learning is a proven technique in 2D computer vision to leverage the large amount of data available and achieve high performance.
In this work, we tackle the case of real-time 3D semantic segmentation of sparse autonomous driving LiDAR scans.
We introduce a new pre-training task: coarse label pre-training, also called COLA.
arXiv Detail & Related papers (2022-02-14T17:19:23Z) - Multi-domain semantic segmentation with overlapping labels [1.4120796122384087]
We propose a principled method for seamless learning on datasets with overlapping classes based on partial labels and probabilistic loss.
Our method achieves competitive within-dataset and cross-dataset generalization, as well as ability to learn visual concepts which are not separately labeled in any of the training datasets.
arXiv Detail & Related papers (2021-08-25T13:25:41Z) - Joining datasets via data augmentation in the label space for neural
networks [6.036150783745836]
We propose a new technique leveraging artificially created knowledge graph, recurrent neural networks and policy gradient that successfully achieve the dataset joining in the label space.
Empirical results on both image and text classification justify the validity of our approach.
arXiv Detail & Related papers (2021-06-17T06:08:11Z) - Adversarial Knowledge Transfer from Unlabeled Data [62.97253639100014]
We present a novel Adversarial Knowledge Transfer framework for transferring knowledge from internet-scale unlabeled data to improve the performance of a classifier.
An important novel aspect of our method is that the unlabeled source data can be of different classes from those of the labeled target data, and there is no need to define a separate pretext task.
arXiv Detail & Related papers (2020-08-13T08:04:27Z) - Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method.
Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
arXiv Detail & Related papers (2020-06-22T08:05:25Z) - Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer
Labels [77.65554439859967]
We propose a weakly supervised point cloud segmentation approach which requires only a tiny fraction of points to be labelled in the training stage.
Experiments are done on three public datasets with different degrees of weak supervision.
arXiv Detail & Related papers (2020-04-08T16:14:41Z) - Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation
on Point Clouds [67.0904905172941]
We propose a weakly supervised approach to predict point-level results using weak labels on 3D point clouds.
To the best of our knowledge, this is the first method that uses cloud-level weak labels on raw 3D space to train a point cloud semantic segmentation network.
arXiv Detail & Related papers (2020-03-29T14:13:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.