Conditional Image Generation with Pretrained Generative Model
- URL: http://arxiv.org/abs/2312.13253v1
- Date: Wed, 20 Dec 2023 18:27:53 GMT
- Title: Conditional Image Generation with Pretrained Generative Model
- Authors: Rajesh Shrestha, Bowen Xie
- Abstract summary: diffusion models have gained popularity for their ability to generate higher-quality images in comparison to GAN models.
These models require a huge amount of data, computational resources, and meticulous tuning for successful training.
We propose methods to leverage pre-trained unconditional diffusion models with additional guidance for the purpose of conditional image generative.
- Score: 1.4685355149711303
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In recent years, diffusion models have gained popularity for their ability to
generate higher-quality images in comparison to GAN models. However, like any
other large generative models, these models require a huge amount of data,
computational resources, and meticulous tuning for successful training. This
poses a significant challenge, rendering it infeasible for most individuals. As
a result, the research community has devised methods to leverage pre-trained
unconditional diffusion models with additional guidance for the purpose of
conditional image generative. These methods enable conditional image
generations on diverse inputs and, most importantly, circumvent the need for
training the diffusion model. In this paper, our objective is to reduce the
time-required and computational overhead introduced by the addition of guidance
in diffusion models -- while maintaining comparable image quality. We propose a
set of methods based on our empirical analysis, demonstrating a reduction in
computation time by approximately threefold.
Related papers
- Fast constrained sampling in pre-trained diffusion models [77.21486516041391]
Diffusion models have dominated the field of large, generative image models.
We propose an algorithm for fast-constrained sampling in large pre-trained diffusion models.
arXiv Detail & Related papers (2024-10-24T14:52:38Z) - Upsample Guidance: Scale Up Diffusion Models without Training [0.0]
We introduce upsample guidance, a technique that adapts pretrained diffusion model to generate higher-resolution images.
Remarkably, this technique does not necessitate any additional training or relying on external models.
We demonstrate that upsample guidance can be applied to various models, such as pixel-space, latent space, and video diffusion models.
arXiv Detail & Related papers (2024-04-02T07:49:08Z) - Training Class-Imbalanced Diffusion Model Via Overlap Optimization [55.96820607533968]
Diffusion models trained on real-world datasets often yield inferior fidelity for tail classes.
Deep generative models, including diffusion models, are biased towards classes with abundant training images.
We propose a method based on contrastive learning to minimize the overlap between distributions of synthetic images for different classes.
arXiv Detail & Related papers (2024-02-16T16:47:21Z) - Large-scale Reinforcement Learning for Diffusion Models [30.164571425479824]
Text-to-image diffusion models are susceptible to implicit biases that arise from web-scale text-image training pairs.
We present an effective scalable algorithm to improve diffusion models using Reinforcement Learning (RL)
We show how our approach substantially outperforms existing methods for aligning diffusion models with human preferences.
arXiv Detail & Related papers (2024-01-20T08:10:43Z) - Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional
Image Synthesis [62.07413805483241]
Steered Diffusion is a framework for zero-shot conditional image generation using a diffusion model trained for unconditional generation.
We present experiments using steered diffusion on several tasks including inpainting, colorization, text-guided semantic editing, and image super-resolution.
arXiv Detail & Related papers (2023-09-30T02:03:22Z) - Conditional Generation from Unconditional Diffusion Models using
Denoiser Representations [94.04631421741986]
We propose adapting pre-trained unconditional diffusion models to new conditions using the learned internal representations of the denoiser network.
We show that augmenting the Tiny ImageNet training set with synthetic images generated by our approach improves the classification accuracy of ResNet baselines by up to 8%.
arXiv Detail & Related papers (2023-06-02T20:09:57Z) - Extracting Training Data from Diffusion Models [77.11719063152027]
We show that diffusion models memorize individual images from their training data and emit them at generation time.
With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models.
We train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy.
arXiv Detail & Related papers (2023-01-30T18:53:09Z) - On Distillation of Guided Diffusion Models [94.95228078141626]
We propose an approach to distilling classifier-free guided diffusion models into models that are fast to sample from.
For standard diffusion models trained on the pixelspace, our approach is able to generate images visually comparable to that of the original model.
For diffusion models trained on the latent-space (e.g., Stable Diffusion), our approach is able to generate high-fidelity images using as few as 1 to 4 denoising steps.
arXiv Detail & Related papers (2022-10-06T18:03:56Z) - Diffusion Models: A Comprehensive Survey of Methods and Applications [10.557289965753437]
Diffusion models are a class of deep generative models that have shown impressive results on various tasks with dense theoretical founding.
Recent studies have shown great enthusiasm on improving the performance of diffusion model.
arXiv Detail & Related papers (2022-09-02T02:59:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.