Learning High-Resolution Domain-Specific Representations with a GAN Generator

• URL: http://arxiv.org/abs/2006.10451v1
• Date: Thu, 18 Jun 2020 11:57:18 GMT
• Title: Learning High-Resolution Domain-Specific Representations with a GAN Generator
• Authors: Danil Galeev, Konstantin Sofiiuk, Danila Rukhovich, Mikhail Romanov, Olga Barinova, Anton Konushin
• Abstract summary: We show that representations learnt by a GAN generator can be easily projected onto semantic segmentation map using a lightweight decoder. We propose LayerMatch scheme for approximating the representation of a GAN generator that can be used for unsupervised domain-specific pretraining. We find that the use of LayerMatch-pretrained backbone leads to superior accuracy compared to standard supervised pretraining on ImageNet.
• Score: 5.8720142291102135
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years generative models of visual data have made a great progress, and now they are able to produce images of high quality and diversity. In this work we study representations learnt by a GAN generator. First, we show that these representations can be easily projected onto semantic segmentation map using a lightweight decoder. We find that such semantic projection can be learnt from just a few annotated images. Based on this finding, we propose LayerMatch scheme for approximating the representation of a GAN generator that can be used for unsupervised domain-specific pretraining. We consider the semi-supervised learning scenario when a small amount of labeled data is available along with a large unlabeled dataset from the same domain. We find that the use of LayerMatch-pretrained backbone leads to superior accuracy compared to standard supervised pretraining on ImageNet. Moreover, this simple approach also outperforms recent semi-supervised semantic segmentation methods that use both labeled and unlabeled data during training. Source code for reproducing our experiments will be available at the time of publication.

Related papers

• Hybrid diffusion models: combining supervised and generative pretraining for label-efficient fine-tuning of segmentation models [55.2480439325792]
  We propose a new pretext task, which is to perform simultaneously image denoising and mask prediction on the first domain. We show that fine-tuning a model pretrained using this approach leads to better results than fine-tuning a similar model trained using either supervised or unsupervised pretraining.
  arXiv  Detail & Related papers  (2024-08-06T20:19:06Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• Task Specific Pretraining with Noisy Labels for Remote Sensing Image Segmentation [18.598405597933752]
  Self-supervision provides remote sensing a tool to reduce the amount of exact, human-crafted geospatial annotations. In this work, we propose to exploit noisy semantic segmentation maps for model pretraining. The results from two datasets indicate the effectiveness of task-specific supervised pretraining with noisy labels.
  arXiv  Detail & Related papers  (2024-02-25T18:01:42Z)
• A Semi-Paired Approach For Label-to-Image Translation [6.888253564585197]
  We introduce the first semi-supervised (semi-paired) framework for label-to-image translation. In the semi-paired setting, the model has access to a small set of paired data and a larger set of unpaired images and labels. We propose a training algorithm for this shared network, and we present a rare classes sampling algorithm to focus on under-represented classes.
  arXiv  Detail & Related papers  (2023-06-23T16:13:43Z)
• Enhancing Self-Supervised Learning for Remote Sensing with Elevation Data: A Case Study with Scarce And High Level Semantic Labels [1.534667887016089]
  This work proposes a hybrid unsupervised and supervised learning method to pre-train models applied in Earth observation downstream tasks. We combine a contrastive approach to pre-train models with a pixel-wise regression pre-text task to predict coarse elevation maps.
  arXiv  Detail & Related papers  (2023-04-13T23:01:11Z)
• Extracting Semantic Knowledge from GANs with Unsupervised Learning [65.32631025780631]
  Generative Adversarial Networks (GANs) encode semantics in feature maps in a linearly separable form. We propose a novel clustering algorithm, named KLiSH, which leverages the linear separability to cluster GAN's features. KLiSH succeeds in extracting fine-grained semantics of GANs trained on datasets of various objects.
  arXiv  Detail & Related papers  (2022-11-30T03:18:16Z)
• BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations [89.42397034542189]
  We synthesize a large labeled dataset via a generative adversarial network (GAN) We take image samples from the class-conditional generative model BigGAN trained on ImageNet, and manually annotate 5 images per class, for all 1k classes. We create a new ImageNet benchmark by labeling an additional set of 8k real images and evaluate segmentation performance in a variety of settings.
  arXiv  Detail & Related papers  (2022-01-12T20:28:34Z)
• Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization [112.68171734288237]
  We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels. We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images. We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
  arXiv  Detail & Related papers  (2021-04-12T21:41:25Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.