Label Name is Mantra: Unifying Point Cloud Segmentation across Heterogeneous Datasets

• URL: http://arxiv.org/abs/2303.10585v1
• Date: Sun, 19 Mar 2023 06:14:22 GMT
• Title: Label Name is Mantra: Unifying Point Cloud Segmentation across Heterogeneous Datasets
• Authors: Yixun Liang, Hao He, Shishi Xiao, Hao Lu and Yingcong Chen
• Abstract summary: We propose a principled approach that supports learning from heterogeneous datasets with different label sets. Our idea is to utilize a pre-trained language model to embed discrete labels to a continuous latent space with the help of their label names. Our model outperforms the state-of-the-art by a large margin.
• Score: 17.503843467554592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Point cloud segmentation is a fundamental task in 3D vision that serves a wide range of applications. Although great progresses have been made these years, its practical usability is still limited by the availability of training data. Existing approaches cannot make full use of multiple datasets on hand due to the label mismatch among different datasets. In this paper, we propose a principled approach that supports learning from heterogeneous datasets with different label sets. Our idea is to utilize a pre-trained language model to embed discrete labels to a continuous latent space with the help of their label names. This unifies all labels of different datasets, so that joint training is doable. Meanwhile, classifying points in the continuous 3D space by their vocabulary tokens significantly increase the generalization ability of the model in comparison with existing approaches that have fixed decoder architecture. Besides, we also integrate prompt learning in our framework to alleviate data shifts among different data sources. Extensive experiments demonstrate that our model outperforms the state-of-the-art by a large margin.

Related papers

• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• Scribbles for All: Benchmarking Scribble Supervised Segmentation Across Datasets [51.74296438621836]
  We introduce Scribbles for All, a label and training data generation algorithm for semantic segmentation trained on scribble labels. The main limitation of scribbles as source for weak supervision is the lack of challenging datasets for scribble segmentation. Scribbles for All provides scribble labels for several popular segmentation datasets and provides an algorithm to automatically generate scribble labels for any dataset with dense annotations.
  arXiv  Detail & Related papers  (2024-08-22T15:29:08Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Teachers in concordance for pseudo-labeling of 3D sequential data [1.1610573589377013]
  We propose to leverage sequences of point clouds to boost the pseudolabeling technique in a teacher-student setup via training multiple teachers. This set of teachers, dubbed Concordance, provides higher quality pseudo-labels for student training than standard methods. Our approach, which uses only 20% manual labels, outperforms some fully supervised methods.
  arXiv  Detail & Related papers  (2022-07-13T09:40:22Z)
• Learning Semantic Segmentation from Multiple Datasets with Label Shifts [101.24334184653355]
  This paper proposes UniSeg, an effective approach to automatically train models across multiple datasets with differing label spaces. Specifically, we propose two losses that account for conflicting and co-occurring labels to achieve better generalization performance in unseen domains.
  arXiv  Detail & Related papers  (2022-02-28T18:55:19Z)
• COLA: COarse LAbel pre-training for 3D semantic segmentation of sparse LiDAR datasets [3.8243923744440926]
  Transfer learning is a proven technique in 2D computer vision to leverage the large amount of data available and achieve high performance. In this work, we tackle the case of real-time 3D semantic segmentation of sparse autonomous driving LiDAR scans. We introduce a new pre-training task: coarse label pre-training, also called COLA.
  arXiv  Detail & Related papers  (2022-02-14T17:19:23Z)
• Multi-domain semantic segmentation with overlapping labels [1.4120796122384087]
  We propose a principled method for seamless learning on datasets with overlapping classes based on partial labels and probabilistic loss. Our method achieves competitive within-dataset and cross-dataset generalization, as well as ability to learn visual concepts which are not separately labeled in any of the training datasets.
  arXiv  Detail & Related papers  (2021-08-25T13:25:41Z)
• Joining datasets via data augmentation in the label space for neural networks [6.036150783745836]
  We propose a new technique leveraging artificially created knowledge graph, recurrent neural networks and policy gradient that successfully achieve the dataset joining in the label space. Empirical results on both image and text classification justify the validity of our approach.
  arXiv  Detail & Related papers  (2021-06-17T06:08:11Z)
• Adversarial Knowledge Transfer from Unlabeled Data [62.97253639100014]
  We present a novel Adversarial Knowledge Transfer framework for transferring knowledge from internet-scale unlabeled data to improve the performance of a classifier. An important novel aspect of our method is that the unlabeled source data can be of different classes from those of the labeled target data, and there is no need to define a separate pretext task.
  arXiv  Detail & Related papers  (2020-08-13T08:04:27Z)
• Few-shot 3D Point Cloud Semantic Segmentation [138.80825169240302]
  We propose a novel attention-aware multi-prototype transductive few-shot point cloud semantic segmentation method. Our proposed method shows significant and consistent improvements compared to baselines in different few-shot point cloud semantic segmentation settings.
  arXiv  Detail & Related papers  (2020-06-22T08:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.