EP-CFG: Energy-Preserving Classifier-Free Guidance

• URL: http://arxiv.org/abs/2412.09966v1
• Date: Fri, 13 Dec 2024 08:49:25 GMT
• Title: EP-CFG: Energy-Preserving Classifier-Free Guidance
• Authors: Kai Zhang, Fujun Luan, Sai Bi, Jianming Zhang,
• Abstract summary: We present EPCFG (Energy-Preserving-Free Guidance), which addresses issues by preserving the energy distribution during conditional prediction. Our method simply rescales the guided output to match that conditional prediction each denoising step, with an optional robust variant for improved artifact suppression.
• Score: 17.356740523778058
• License:
• Abstract: Classifier-free guidance (CFG) is widely used in diffusion models but often introduces over-contrast and over-saturation artifacts at higher guidance strengths. We present EP-CFG (Energy-Preserving Classifier-Free Guidance), which addresses these issues by preserving the energy distribution of the conditional prediction during the guidance process. Our method simply rescales the energy of the guided output to match that of the conditional prediction at each denoising step, with an optional robust variant for improved artifact suppression. Through experiments, we show that EP-CFG maintains natural image quality and preserves details across guidance strengths while retaining CFG's semantic alignment benefits, all with minimal computational overhead.

Related papers

• On the Convergence of DP-SGD with Adaptive Clipping [56.24689348875711]
  Gradient Descent with gradient clipping is a powerful technique for enabling differentially private optimization. This paper provides the first comprehensive convergence analysis of SGD with quantile clipping (QC-SGD) We show how QC-SGD suffers from a bias problem similar to constant-threshold clipped SGD but can be mitigated through a carefully designed quantile and step size schedule.
  arXiv  Detail & Related papers  (2024-12-27T20:29:47Z)
• Contrastive CFG: Improving CFG in Diffusion Models by Contrasting Positive and Negative Concepts [55.298031232672734]
  As-Free Guidance (CFG) has proven effective in conditional diffusion model sampling for improved condition alignment. We present a novel method to enhance negative CFG guidance using contrastive loss.
  arXiv  Detail & Related papers  (2024-11-26T03:29:27Z)
• Rectified Diffusion Guidance for Conditional Generation [62.00207951161297]
  We revisit the theory behind CFG and rigorously confirm that the improper configuration of the combination coefficients (i.e., the widely used summing-to-one version) brings about expectation shift of the generative distribution. We propose ReCFG with a relaxation on the guidance coefficients such that denoising with ReCFG strictly aligns with the diffusion theory. That way the rectified coefficients can be readily pre-computed via traversing the observed data, leaving the sampling speed barely affected.
  arXiv  Detail & Related papers  (2024-10-24T13:41:32Z)
• Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models [27.640009920058187]
  We revisit the CFG update rule and introduce modifications to address this issue. We propose down-weighting the parallel component to achieve high-quality generations without oversaturation. We also introduce a new rescaling momentum method for the CFG update rule based on this insight.
  arXiv  Detail & Related papers  (2024-10-03T12:06:29Z)
• Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention [0.7770029179741429]
  Conditional diffusion models have shown remarkable success in visual content generation. Recent attempts to extend unconditional guidance have relied on techniques, resulting in suboptimal generation quality. We propose Smoothed Energy Guidance (SEG), a novel training- and condition-free approach to enhance image generation.
  arXiv  Detail & Related papers  (2024-08-01T17:59:09Z)
• Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models [44.58960475893552]
  "Adaptive Guidance" (AG) is an efficient variant of computation-Free Guidance (CFG) AG preserves CFG's image quality while reducing by 25%. " LinearAG" offers even cheaper inference at the cost of deviating from the baseline model.
  arXiv  Detail & Related papers  (2023-12-19T17:08:48Z)
• Manifold Preserving Guided Diffusion [121.97907811212123]
  Conditional image generation still faces challenges of cost, generalizability, and the need for task-specific training. We propose Manifold Preserving Guided Diffusion (MPGD), a training-free conditional generation framework.
  arXiv  Detail & Related papers  (2023-11-28T02:08:06Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning [44.880922634512096]
  This paper considers a general setting where the guidance is defined by an (unnormalized) energy function. The main challenge for this setting is that the intermediate guidance during the diffusion sampling procedure is unknown and is hard to estimate. We propose an exact formulation of the intermediate guidance as well as a novel training objective named contrastive energy prediction (CEP) to learn the exact guidance.
  arXiv  Detail & Related papers  (2023-04-25T13:50:41Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.