Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloud

• URL: http://arxiv.org/abs/2108.06230v1
• Date: Fri, 13 Aug 2021 13:29:27 GMT
• Title: Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Cloud
• Authors: Bj\"orn Michele, Alexandre Boulch, Gilles Puy, Renaud Marlet
• Abstract summary: We present the first generative approach for both Zero-Shot Learning (ZSL) and Generalized ZSL (GZSL) on 3D data. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. Our experiments show that our method outperforms strong baselines, which we additionally propose for this task.
• Score: 79.99653758293277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While there has been a number of studies on Zero-Shot Learning (ZSL) for 2D images, its application to 3D data is still recent and scarce, with just a few methods limited to classification. We present the first generative approach for both ZSL and Generalized ZSL (GZSL) on 3D data, that can handle both classification and, for the first time, semantic segmentation. We show that it reaches or outperforms the state of the art on ModelNet40 classification for both inductive ZSL and inductive GZSL. For semantic segmentation, we created three benchmarks for evaluating this new ZSL task, using S3DIS, ScanNet and SemanticKITTI. Our experiments show that our method outperforms strong baselines, which we additionally propose for this task.

Related papers

• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• Visual Self-supervised Learning Scheme for Dense Prediction Tasks on X-ray Images [3.782392436834913]
  Self-supervised learning (SSL) has led to considerable progress in natural language processing (NLP) However, the incorporation of contrastive learning into existing visual SSL models has led to considerable progress, often surpassing supervised counterparts. Here, we focus on dense prediction tasks using security inspection x-ray images to evaluate our proposed model, Segment localization (SegLoc) Based upon the Instance localization (InsLoc) model, SegLoc addresses one of the key challenges of contrastive learning, i.e., false negative pairs of query embeddings.
  arXiv  Detail & Related papers  (2023-10-12T15:42:17Z)
• Zero-Shot Learning by Harnessing Adversarial Samples [52.09717785644816]
  We propose a novel Zero-Shot Learning (ZSL) approach by Harnessing Adversarial Samples (HAS) HAS advances ZSL through adversarial training which takes into account three crucial aspects. We demonstrate the effectiveness of our adversarial samples approach in both ZSL and Generalized Zero-Shot Learning (GZSL) scenarios.
  arXiv  Detail & Related papers  (2023-08-01T06:19:13Z)
• Rethinking Self-Supervised Visual Representation Learning in Pre-training for 3D Human Pose and Shape Estimation [57.206129938611454]
  Self-supervised representation learning (SSL) methods have outperformed the ImageNet classification pre-training for vision tasks such as object detection. We empirically study and analyze the effects of SSL and compare it with other pre-training alternatives for 3DHPSE. Our observations challenge the naive application of the current SSL pre-training to 3DHPSE and relight the value of other data types in the pre-training aspect.
  arXiv  Detail & Related papers  (2023-03-09T16:17:52Z)
• A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends [82.64268080902742]
  Self-supervised learning (SSL) aims to learn discriminative features from unlabeled data without relying on human-annotated labels. SSL has garnered significant attention recently, leading to the development of numerous related algorithms. This paper presents a review of diverse SSL methods, encompassing algorithmic aspects, application domains, three key trends, and open research questions.
  arXiv  Detail & Related papers  (2023-01-13T14:41:05Z)
• Class-Level Confidence Based 3D Semi-Supervised Learning [18.95161296147023]
  We show that unlabeled data class-level confidence can represent the learning status in the 3D imbalanced dataset. Our method significantly outperforms state-of-the-art counterparts for both 3D SSL classification and detection tasks.
  arXiv  Detail & Related papers  (2022-10-18T20:13:28Z)
• LETS-GZSL: A Latent Embedding Model for Time Series Generalized Zero Shot Learning [1.4665304971699262]
  We propose a Latent Embedding for Time Series - GZSL (LETS-GZSL) model that can solve the problem of GZSL for time series classification (TSC) Our framework is able to achieve a harmonic mean value of at least 55% on most datasets except when the number of unseen classes is greater than 3.
  arXiv  Detail & Related papers  (2022-07-25T09:31:22Z)
• Hardness Sampling for Self-Training Based Transductive Zero-Shot Learning [10.764160559530847]
  Transductive zero-shot learning (T-ZSL) which could alleviate the domain shift problem in existing ZSL works, has received much attention recently. We first empirically analyze the roles of unseen-class samples with different degrees of hardness in the training process. We propose two hardness sampling approaches for selecting a subset of diverse and hard samples from a given unseen-class dataset.
  arXiv  Detail & Related papers  (2021-06-01T06:55:19Z)
• Zero-Shot Learning on 3D Point Cloud Objects and Beyond [21.6491982908705]
  We identify some of the challenges and apply 2D Zero-Shot Learning (ZSL) methods in the 3D domain to analyze the performance of existing models. A novel loss function is developed that simultaneously aligns seen semantics with point cloud features. An extensive set of experiments is carried out, establishing state-of-the-art for ZSL and GZSL on synthetic and real datasets.
  arXiv  Detail & Related papers  (2021-04-11T10:04:06Z)
• End-to-end Generative Zero-shot Learning via Few-shot Learning [76.9964261884635]
  State-of-the-art approaches to Zero-Shot Learning (ZSL) train generative nets to synthesize examples conditioned on the provided metadata. We introduce an end-to-end generative ZSL framework that uses such an approach as a backbone and feeds its synthesized output to a Few-Shot Learning algorithm.
  arXiv  Detail & Related papers  (2021-02-08T17:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.