Step Saver: Predicting Minimum Denoising Steps for Diffusion Model Image Generation

• URL: http://arxiv.org/abs/2408.02054v1
• Date: Sun, 4 Aug 2024 15:01:23 GMT
• Title: Step Saver: Predicting Minimum Denoising Steps for Diffusion Model Image Generation
• Authors: Jean Yu, Haim Barad,
• Abstract summary: This paper introduces an innovative NLP model to determine the minimal number of denoising steps required for any given text prompt. It is designed to work seamlessly with the Diffusion model, ensuring that images are produced with superior quality in the shortest possible time.
• Score: 0.6906005491572401
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we introduce an innovative NLP model specifically fine-tuned to determine the minimal number of denoising steps required for any given text prompt. This advanced model serves as a real-time tool that recommends the ideal denoise steps for generating high-quality images efficiently. It is designed to work seamlessly with the Diffusion model, ensuring that images are produced with superior quality in the shortest possible time. Although our explanation focuses on the DDIM scheduler, the methodology is adaptable and can be applied to various other schedulers like Euler, Euler Ancestral, Heun, DPM2 Karras, UniPC, and more. This model allows our customers to conserve costly computing resources by executing the fewest necessary denoising steps to achieve optimal quality in the produced images.

Related papers

• Cost-Aware Routing for Efficient Text-To-Image Generation [19.848723289971208]
  We propose a framework to allow the amount of computation to vary for each prompt, depending on its complexity.<n>We empirically demonstrate on COCO and DiffusionDB that by learning to route to nine already-trained text-to-image models, our approach is able to deliver an average quality that is higher than that achievable by any of these models alone.
  arXiv  Detail & Related papers  (2025-06-17T17:48:50Z)
• Schedule On the Fly: Diffusion Time Prediction for Faster and Better Image Generation [30.60041718224064]
  Multi-step reverse diffusion process can be regarded as a kind of chain-of-thought for generating high-quality images step by step. Time Prediction Diffusion Model (TPDM) employs a plug-and-play Time Prediction Module (TPM) that predicts the next noise level based on current latent features at each denoising step. TPDM achieves an aesthetic score of 5.44 and a human preference score (HPS) of 29.59, while using around 50% fewer denoising steps to achieve better performance.
  arXiv  Detail & Related papers  (2024-12-02T08:05:26Z)
• PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution [87.89013794655207]
  Diffusion-based image super-resolution (SR) models have shown superior performance at the cost of multiple denoising steps. We propose a novel post-training quantization approach with adaptive scale in one-step diffusion (OSD) image SR, PassionSR. Our PassionSR achieves significant advantages over recent leading low-bit quantization methods for image SR.
  arXiv  Detail & Related papers  (2024-11-26T04:49:42Z)
• Model-Agnostic Human Preference Inversion in Diffusion Models [31.992947353231564]
  We propose a novel sampling design to achieve high-quality one-step image generation aligning with human preferences. Our approach, Prompt Adaptive Human Preference Inversion (PAHI), optimize the noise distributions for each prompt based on human preferences. Our experiments showcase that the tailored noise distributions significantly improve image quality with only a marginal increase in computational cost.
  arXiv  Detail & Related papers  (2024-04-01T03:18:12Z)
• Bellman Optimal Stepsize Straightening of Flow-Matching Models [14.920260435839992]
  This paper introduces Bellman Optimal Stepsize Straightening (BOSS) technique for distilling flow-matching generative models. BOSS aims specifically for a few-step efficient image sampling while adhering to a computational budget constraint. Our results reveal that BOSS achieves substantial gains in efficiency while maintaining competitive sample quality.
  arXiv  Detail & Related papers  (2023-12-27T05:20:20Z)
• AdaDiff: Adaptive Step Selection for Fast Diffusion [88.8198344514677]
  We introduce AdaDiff, a framework designed to learn instance-specific step usage policies. AdaDiff is optimized using a policy gradient method to maximize a carefully designed reward function. Our approach achieves similar results in terms of visual quality compared to the baseline using a fixed 50 denoising steps.
  arXiv  Detail & Related papers  (2023-11-24T11:20:38Z)
• BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping [64.54271680071373]
  Diffusion models have demonstrated excellent potential for generating diverse images. Knowledge distillation has been recently proposed as a remedy that can reduce the number of inference steps to one or a few. We present a novel technique called BOOT, that overcomes limitations with an efficient data-free distillation algorithm.
  arXiv  Detail & Related papers  (2023-06-08T20:30:55Z)
• CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution [91.56337748920662]
  Diffusion probabilistic models (DPM) have been widely adopted in image-to-image translation. We propose a simple but non-trivial DPM-based super-resolution post-process framework,i.e., cDPMSR. Our method surpasses prior attempts on both qualitative and quantitative results.
  arXiv  Detail & Related papers  (2023-02-14T15:13:33Z)
• ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models [22.84873720309945]
  We propose Iterative Latent Variable Refinement (ILVR) to guide the generative process in DDPM to generate high-quality images. The proposed ILVR method generates high-quality images while controlling the generation.
  arXiv  Detail & Related papers  (2021-08-06T04:43:13Z)
• Learning to Efficiently Sample from Diffusion Probabilistic Models [49.58748345998702]
  Denoising Diffusion Probabilistic Models (DDPMs) can yield high-fidelity samples and competitive log-likelihoods across a range of domains. We introduce an exact dynamic programming algorithm that finds the optimal discrete time schedules for any pre-trained DDPM.
  arXiv  Detail & Related papers  (2021-06-07T17:15:07Z)
• Noise Estimation for Generative Diffusion Models [91.22679787578438]
  In this work, we present a simple and versatile learning scheme that can adjust the noise parameters for any given number of steps. Our approach comes at a negligible computation cost.
  arXiv  Detail & Related papers  (2021-04-06T15:46:16Z)
• Learning an Adaptive Model for Extreme Low-light Raw Image Processing [5.706764509663774]
  We propose an adaptive low-light raw image enhancement network to improve image quality. The proposed method has the lowest Noise Level Estimation (NLE) score compared with the state-of-the-art low-light algorithms. The potential application in video processing is briefly discussed.
  arXiv  Detail & Related papers  (2020-04-22T09:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.