A Unified Generative Adversarial Network Training via Self-Labeling and Self-Attention

• URL: http://arxiv.org/abs/2106.09914v1
• Date: Fri, 18 Jun 2021 04:40:26 GMT
• Title: A Unified Generative Adversarial Network Training via Self-Labeling and Self-Attention
• Authors: Tomoki Watanabe, Paolo Favaro
• Abstract summary: We propose a novel GAN training scheme that can handle any level of labeling in a unified manner. Our scheme introduces a form of artificial labeling that can incorporate manually defined labels, when available. We evaluate our approach on CIFAR-10, STL-10 and SVHN, and show that both self-labeling and self-attention consistently improve the quality of generated data.
• Score: 38.31735499785227
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel GAN training scheme that can handle any level of labeling in a unified manner. Our scheme introduces a form of artificial labeling that can incorporate manually defined labels, when available, and induce an alignment between them. To define the artificial labels, we exploit the assumption that neural network generators can be trained more easily to map nearby latent vectors to data with semantic similarities, than across separate categories. We use generated data samples and their corresponding artificial conditioning labels to train a classifier. The classifier is then used to self-label real data. To boost the accuracy of the self-labeling, we also use the exponential moving average of the classifier. However, because the classifier might still make mistakes, especially at the beginning of the training, we also refine the labels through self-attention, by using the labeling of real data samples only when the classifier outputs a high classification probability score. We evaluate our approach on CIFAR-10, STL-10 and SVHN, and show that both self-labeling and self-attention consistently improve the quality of generated data. More surprisingly, we find that the proposed scheme can even outperform class-conditional GANs.

Related papers

• Reduction-based Pseudo-label Generation for Instance-dependent Partial Label Learning [41.345794038968776]
  We propose to leverage reduction-based pseudo-labels to alleviate the influence of incorrect candidate labels. We show that reduction-based pseudo-labels exhibit greater consistency with the Bayes optimal classifier compared to pseudo-labels directly generated from the predictive model.
  arXiv  Detail & Related papers  (2024-10-28T07:32:20Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Label distribution learning via label correlation grid [9.340734188957727]
  We propose a textbfLabel textbfCorrelation textbfGrid (LCG) to model the uncertainty of label relationships. Our network learns the LCG to accurately estimate the label distribution for each instance.
  arXiv  Detail & Related papers  (2022-10-15T03:58:15Z)
• Distribution-Aware Semantics-Oriented Pseudo-label for Imbalanced Semi-Supervised Learning [80.05441565830726]
  This paper addresses imbalanced semi-supervised learning, where heavily biased pseudo-labels can harm the model performance. We propose a general pseudo-labeling framework to address the bias motivated by this observation. We term the novel pseudo-labeling framework for imbalanced SSL as Distribution-Aware Semantics-Oriented (DASO) Pseudo-label.
  arXiv  Detail & Related papers  (2021-06-10T11:58:25Z)
• Cycle Self-Training for Domain Adaptation [85.14659717421533]
  Cycle Self-Training (CST) is a principled self-training algorithm that enforces pseudo-labels to generalize across domains. CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over prior state-of-the-arts in standard UDA benchmarks.
  arXiv  Detail & Related papers  (2021-03-05T10:04:25Z)
• Sinkhorn Label Allocation: Semi-Supervised Classification via Annealed Self-Training [38.81973113564937]
  Self-training is a standard approach to semi-supervised learning where the learner's own predictions on unlabeled data are used as supervision during training. In this paper, we reinterpret this label assignment problem as an optimal transportation problem between examples and classes. We demonstrate the effectiveness of our algorithm on the CIFAR-10, CIFAR-100, and SVHN datasets in comparison with FixMatch, a state-of-the-art self-training algorithm.
  arXiv  Detail & Related papers  (2021-02-17T08:23:15Z)
• Label Confusion Learning to Enhance Text Classification Models [3.0251266104313643]
  Label Confusion Model (LCM) learns label confusion to capture semantic overlap among labels. LCM can generate a better label distribution to replace the original one-hot label vector. experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models.
  arXiv  Detail & Related papers  (2020-12-09T11:34:35Z)
• Semi-Supervised Speech Recognition via Graph-based Temporal Classification [59.58318952000571]
  Semi-supervised learning has demonstrated promising results in automatic speech recognition by self-training. The effectiveness of this approach largely relies on the pseudo-label accuracy. Alternative ASR hypotheses of an N-best list can provide more accurate labels for an unlabeled speech utterance.
  arXiv  Detail & Related papers  (2020-10-29T14:56:56Z)
• Probabilistic Decoupling of Labels in Classification [4.865747672937677]
  We develop a principled, probabilistic, unified approach to non-standard classification tasks. We train a classifier on the given labels to predict the label-distribution. We then infer the underlying class-distributions by variationally optimizing a model of label-class transitions.
  arXiv  Detail & Related papers  (2020-06-16T10:07:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.