Don't Play Favorites: Minority Guidance for Diffusion Models

• URL: http://arxiv.org/abs/2301.12334v2
• Date: Mon, 26 Feb 2024 15:38:28 GMT
• Title: Don't Play Favorites: Minority Guidance for Diffusion Models
• Authors: Soobin Um, Suhyeon Lee, Jong Chul Ye
• Abstract summary: We present a novel framework that can make the generation process of the diffusion models focus on the minority samples. We develop minority guidance, a sampling technique that can guide the generation process toward regions with desired likelihood levels.
• Score: 59.75996752040651
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore the problem of generating minority samples using diffusion models. The minority samples are instances that lie on low-density regions of a data manifold. Generating a sufficient number of such minority instances is important, since they often contain some unique attributes of the data. However, the conventional generation process of the diffusion models mostly yields majority samples (that lie on high-density regions of the manifold) due to their high likelihoods, making themselves ineffective and time-consuming for the minority generating task. In this work, we present a novel framework that can make the generation process of the diffusion models focus on the minority samples. We first highlight that Tweedie's denoising formula yields favorable results for majority samples. The observation motivates us to introduce a metric that describes the uniqueness of a given sample. To address the inherent preference of the diffusion models w.r.t. the majority samples, we further develop minority guidance, a sampling technique that can guide the generation process toward regions with desired likelihood levels. Experiments on benchmark real datasets demonstrate that our minority guidance can greatly improve the capability of generating high-quality minority samples over existing generative samplers. We showcase that the performance benefit of our framework persists even in demanding real-world scenarios such as medical imaging, further underscoring the practical significance of our work. Code is available at https://github.com/soobin-um/minority-guidance.

Related papers

• MinorityPrompt: Text to Minority Image Generation via Prompt Optimization [57.319845580050924]
  We investigate the generation of minority samples using pretrained text-to-image (T2I) latent diffusion models. We develop an online prompt optimization framework that can encourage the emergence of desired properties. We then tailor this generic prompt into a specialized solver that promotes the generation of minority features.
  arXiv  Detail & Related papers  (2024-10-10T11:56:09Z)
• Self-Guided Generation of Minority Samples Using Diffusion Models [57.319845580050924]
  We present a novel approach for generating minority samples that live on low-density regions of a data manifold. Our framework is built upon diffusion models, leveraging the principle of guided sampling. Experiments on benchmark real datasets demonstrate that our approach can greatly improve the capability of creating realistic low-likelihood minority instances.
  arXiv  Detail & Related papers  (2024-07-16T10:03:29Z)
• Generative Oversampling for Imbalanced Data via Majority-Guided VAE [15.93867386081279]
  We propose a novel over-sampling model, called Majority-Guided VAE(MGVAE), which generates new minority samples under the guidance of a majority-based prior. In this way, the newly generated minority samples can inherit the diversity and richness of the majority ones, thus mitigating overfitting in downstream tasks.
  arXiv  Detail & Related papers  (2023-02-14T06:35:23Z)
• Reducing Training Sample Memorization in GANs by Training with Memorization Rejection [80.0916819303573]
  We propose rejection memorization, a training scheme that rejects generated samples that are near-duplicates of training samples during training. Our scheme is simple, generic and can be directly applied to any GAN architecture.
  arXiv  Detail & Related papers  (2022-10-21T20:17:50Z)
• Generating High Fidelity Data from Low-density Regions using Diffusion Models [15.819414178363571]
  We leverage diffusion process based generative models to synthesize novel images from low-density regions. We modify the sampling process to guide it towards low-density regions while simultaneously maintaining the fidelity of synthetic data.
  arXiv  Detail & Related papers  (2022-03-31T17:56:25Z)
• Saliency Grafting: Innocuous Attribution-Guided Mixup with Calibrated Label Mixing [104.630875328668]
  Mixup scheme suggests mixing a pair of samples to create an augmented training sample. We present a novel, yet simple Mixup-variant that captures the best of both worlds.
  arXiv  Detail & Related papers  (2021-12-16T11:27:48Z)
• Sampling from Arbitrary Functions via PSD Models [55.41644538483948]
  We take a two-step approach by first modeling the probability distribution and then sampling from that model. We show that these models can approximate a large class of densities concisely using few evaluations, and present a simple algorithm to effectively sample from these models.
  arXiv  Detail & Related papers  (2021-10-20T12:25:22Z)
• Improved Denoising Diffusion Probabilistic Models [4.919647298882951]
  We show that DDPMs can achieve competitive log-likelihoods while maintaining high sample quality. We also find that learning variances of the reverse diffusion process allows sampling with an order of magnitude fewer forward passes. We show that the sample quality and likelihood of these models scale smoothly with model capacity and training compute, making them easily scalable.
  arXiv  Detail & Related papers  (2021-02-18T23:44:17Z)
• Counterfactual-based minority oversampling for imbalanced classification [11.140929092818235]
  A key challenge of oversampling in imbalanced classification is that the generation of new minority samples often neglects the usage of majority classes. We present a new oversampling framework based on the counterfactual theory.
  arXiv  Detail & Related papers  (2020-08-21T14:13:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.