Data Cleansing for GANs

• URL: http://arxiv.org/abs/2504.00603v1
• Date: Tue, 01 Apr 2025 10:02:37 GMT
• Title: Data Cleansing for GANs
• Authors: Naoyuki Terashita, Hiroki Ohashi, Satoshi Hara,
• Abstract summary: One effective strategy that applies to any machine learning task is identifying harmful instances, whose removal improves the performance.<n>Previous approaches require that the absence of a training instance directly affects the parameters.<n>We propose an instance evaluation scheme that measures the harmfulness of each training instance based on how a GAN evaluation metric is expected to change by the instance's removal.
• Score: 12.466874710578612
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As the application of generative adversarial networks (GANs) expands, it becomes increasingly critical to develop a unified approach that improves performance across various generative tasks. One effective strategy that applies to any machine learning task is identifying harmful instances, whose removal improves the performance. While previous studies have successfully estimated these harmful training instances in supervised settings, their approaches are not easily applicable to GANs. The challenge lies in two requirements of the previous approaches that do not apply to GANs. First, previous approaches require that the absence of a training instance directly affects the parameters. However, in the training for GANs, the instances do not directly affect the generator's parameters since they are only fed into the discriminator. Second, previous approaches assume that the change in loss directly quantifies the harmfulness of the instance to a model's performance, while common types of GAN losses do not always reflect the generative performance. To overcome the first challenge, we propose influence estimation methods that use the Jacobian of the generator's gradient with respect to the discriminator's parameters (and vice versa). Such a Jacobian represents the indirect effect between two models: how removing an instance from the discriminator's training changes the generator's parameters. Second, we propose an instance evaluation scheme that measures the harmfulness of each training instance based on how a GAN evaluation metric (e.g., Inception score) is expected to change by the instance's removal. Furthermore, we demonstrate that removing the identified harmful instances significantly improves the generative performance on various GAN evaluation metrics.

Related papers

• Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networks [5.107720313575234]
  This work aims to prevent the generation of outputs containing undesired features from a pre-trained Generative Adversarial Network (GAN)<n>Our proposed two-stage method, known as 'Adapt-then-Unlearn,' excels at unlearning such undesirable features while also maintaining the quality of generated samples.<n>To the best of our knowledge, our approach stands as the first method addressing unlearning within the realm of high-fidelity GANs.
  arXiv  Detail & Related papers  (2023-09-25T11:36:20Z)
• Training Discrete Deep Generative Models via Gapped Straight-Through Estimator [72.71398034617607]
  We propose a Gapped Straight-Through ( GST) estimator to reduce the variance without incurring resampling overhead. This estimator is inspired by the essential properties of Straight-Through Gumbel-Softmax. Experiments demonstrate that the proposed GST estimator enjoys better performance compared to strong baselines on two discrete deep generative modeling tasks.
  arXiv  Detail & Related papers  (2022-06-15T01:46:05Z)
• Augmentation-Aware Self-Supervision for Data-Efficient GAN Training [68.81471633374393]
  Training generative adversarial networks (GANs) with limited data is challenging because the discriminator is prone to overfitting. We propose a novel augmentation-aware self-supervised discriminator that predicts the augmentation parameter of the augmented data. We compare our method with state-of-the-art (SOTA) methods using the class-conditional BigGAN and unconditional StyleGAN2 architectures.
  arXiv  Detail & Related papers  (2022-05-31T10:35:55Z)
• Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
  Learning from implicit feedback is challenging because of the difficult nature of the one-class problem. Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem. We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
  arXiv  Detail & Related papers  (2021-05-11T03:38:16Z)
• Demonstrating the Evolution of GANs through t-SNE [0.4588028371034407]
  Evolutionary algorithms, such as COEGAN, were recently proposed as a solution to improve the GAN training. In this work, we propose an evaluation method based on t-distributed Neighbour Embedding (t-SNE) to assess the progress of GANs. A metric based on the resulting t-SNE maps and the Jaccard index is proposed to represent the model quality.
  arXiv  Detail & Related papers  (2021-01-31T20:07:08Z)
• The Hidden Tasks of Generative Adversarial Networks: An Alternative Perspective on GAN Training [1.964574177805823]
  We present an alternative perspective on the training of generative adversarial networks (GANs) We show that the training step for a GAN generator decomposes into two implicit sub-problems. We experimentally validate our main theoretical result and discuss implications for alternative training methods.
  arXiv  Detail & Related papers  (2021-01-28T08:17:29Z)
• Influence Estimation for Generative Adversarial Networks [0.4014524824655105]
  Previous approaches require that the absence of a training instance directly affects the loss value. We propose a novel evaluation scheme, in which we assess harmfulness of each training instance on the basis of how GAN evaluation metric is expect to change.
  arXiv  Detail & Related papers  (2021-01-20T23:55:54Z)
• A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation [53.8171136907856]
  We introduce a set of simple yet effective data augmentation strategies dubbed cutoff. cutoff relies on sampling consistency and thus adds little computational overhead. cutoff consistently outperforms adversarial training and achieves state-of-the-art results on the IWSLT2014 German-English dataset.
  arXiv  Detail & Related papers  (2020-09-29T07:08:35Z)
• Estimating Generalization under Distribution Shifts via Domain-Invariant Representations [75.74928159249225]
  We use a set of domain-invariant predictors as a proxy for the unknown, true target labels. The error of the resulting risk estimate depends on the target risk of the proxy model.
  arXiv  Detail & Related papers  (2020-07-06T17:21:24Z)
• Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator [85.68825725223873]
  Generative Adversarial Networks (GANs) have shown great promise in modeling high dimensional data. We introduce the Discriminator Contrastive Divergence, which is well motivated by the property of WGAN's discriminator. We demonstrate the benefits of significant improved generation on both synthetic data and several real-world image generation benchmarks.
  arXiv  Detail & Related papers  (2020-04-05T01:50:16Z)
• Discriminative Adversarial Search for Abstractive Summarization [29.943949944682196]
  We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS) DAS has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. We investigate the effectiveness of the proposed approach on the task of Abstractive Summarization.
  arXiv  Detail & Related papers  (2020-02-24T17:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.