Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms

• URL: http://arxiv.org/abs/2502.07849v2
• Date: Thu, 22 May 2025 14:59:12 GMT
• Title: Classifier-Free Guidance: From High-Dimensional Analysis to Generalized Guidance Forms
• Authors: Krunoslav Lehman Pavasovic, Jakob Verbeek, Giulio Biroli, Marc Mezard,
• Abstract summary: We show that CFG accurately reproduces the target distribution in sufficiently high and infinite dimensions.<n>We show that there is a large family of guidances enjoying this property, in particular nonlinear CFG generalizations.<n>Our findings are validated with experiments on class-conditional and text-to-image generation using state-of-the-art diffusion and flow-matching models.
• Score: 22.44946627454133
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classifier-Free Guidance (CFG) is a widely adopted technique in diffusion and flow-based generative models, enabling high-quality conditional generation. A key theoretical challenge is characterizing the distribution induced by CFG, particularly in high-dimensional settings relevant to real-world data. Previous works have shown that CFG modifies the target distribution, steering it towards a distribution sharper than the target one, more shifted towards the boundary of the class. In this work, we provide a high-dimensional analysis of CFG, showing that these distortions vanish as the data dimension grows. We present a blessing-of-dimensionality result demonstrating that in sufficiently high and infinite dimensions, CFG accurately reproduces the target distribution. Using our high-dimensional theory, we show that there is a large family of guidances enjoying this property, in particular non-linear CFG generalizations. We study a simple non-linear power-law version, for which we demonstrate improved robustness, sample fidelity and diversity. Our findings are validated with experiments on class-conditional and text-to-image generation using state-of-the-art diffusion and flow-matching models.

Related papers

• Conditional Diffusion Models with Classifier-Free Gibbs-like Guidance [19.83064246586143]
  CFG is a technique for improving conditional diffusion models by linearly combining the outputs of conditional and unconditional denoisers.<n>While CFG enhances visual quality and improves alignment with prompts, it often reduces sample diversity.<n>We propose a Gibbs-like sampling procedure to draw samples from the desired tilted distribution.
  arXiv  Detail & Related papers  (2025-05-27T12:27:33Z)
• Diffusion Models without Classifier-free Guidance [41.59396565229466]
  Model-guidance (MG) is a novel objective for training diffusion model addresses and removes commonly used guidance (CFG)<n>Our innovative approach transcends the standard modeling and incorporates the posterior probability of conditions.<n>Our method significantly accelerates the training process, doubles inference speed, and achieve exceptional quality that parallel surpass even concurrent diffusion models with CFG.
  arXiv  Detail & Related papers  (2025-02-17T18:59:50Z)
• Nested Annealed Training Scheme for Generative Adversarial Networks [54.70743279423088]
  This paper focuses on a rigorous mathematical theoretical framework: the composite-functional-gradient GAN (CFG)<n>We reveal the theoretical connection between the CFG model and score-based models.<n>We find that the training objective of the CFG discriminator is equivalent to finding an optimal D(x)
  arXiv  Detail & Related papers  (2025-01-20T07:44:09Z)
• A New Formulation of Lipschitz Constrained With Functional Gradient Learning for GANs [52.55025869932486]
  This paper introduces a promising alternative method for training Generative Adversarial Networks (GANs) on large-scale datasets with clear theoretical guarantees.<n>We propose a novel Lipschitz-constrained Functional Gradient GANs learning (Li-CFG) method to stabilize the training of GAN.<n>We demonstrate that the neighborhood size of the latent vector can be reduced by increasing the norm of the discriminator gradient.
  arXiv  Detail & Related papers  (2025-01-20T02:48:07Z)
• Contrastive CFG: Improving CFG in Diffusion Models by Contrasting Positive and Negative Concepts [55.298031232672734]
  As-Free Guidance (CFG) has proven effective in conditional diffusion model sampling for improved condition alignment. We present a novel method to enhance negative CFG guidance using contrastive loss.
  arXiv  Detail & Related papers  (2024-11-26T03:29:27Z)
• CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models [52.29804282879437]
  CFG++ is a novel approach that tackles the offmanifold challenges inherent to traditional CFG. It offers better inversion-to-image generation, invertibility, smaller guidance scales, reduced mode collapse, etc. It can be easily integrated into high-order diffusion solvers and naturally extends to distilled diffusion models.
  arXiv  Detail & Related papers  (2024-06-12T10:40:10Z)
• Distributional Reduction: Unifying Dimensionality Reduction and Clustering with Gromov-Wasserstein [56.62376364594194]
  Unsupervised learning aims to capture the underlying structure of potentially large and high-dimensional datasets. In this work, we revisit these approaches under the lens of optimal transport and exhibit relationships with the Gromov-Wasserstein problem. This unveils a new general framework, called distributional reduction, that recovers DR and clustering as special cases and allows addressing them jointly within a single optimization problem.
  arXiv  Detail & Related papers  (2024-02-03T19:00:19Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Boosted Control Functions: Distribution generalization and invariance in confounded models [10.503777692702952]
  We introduce a strong notion of invariance that allows for distribution generalization even in the presence of nonlinear, non-identifiable structural functions.<n>We propose the ControlTwicing algorithm to estimate the Boosted Control Function (BCF) using flexible machine-learning techniques.
  arXiv  Detail & Related papers  (2023-10-09T15:43:46Z)
• Consistency Regularization for Generalizable Source-free Domain Adaptation [62.654883736925456]
  Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset. Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets. We propose a consistency regularization framework to develop a more generalizable SFDA method.
  arXiv  Detail & Related papers  (2023-08-03T07:45:53Z)
• On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data [26.310275682709776]
  Various logit-adjusted parameterizations of the cross-entropy (CE) loss have been proposed as alternatives to weighted CE large models on labelimbalanced data. We show that logit-adjusted parameterizations can be appropriately tuned to learn to learn irrespective of the minority imbalance ratio.
  arXiv  Detail & Related papers  (2023-03-14T03:04:37Z)
• Causal Graph Discovery from Self and Mutually Exciting Time Series [10.410454851418548]
  We develop a non-asymptotic recovery guarantee and quantifiable uncertainty by solving a linear program. We demonstrate the effectiveness of our approach in recovering highly interpretable causal DAGs over Sepsis Associated Derangements (SADs)
  arXiv  Detail & Related papers  (2023-01-26T16:15:27Z)
• Pseudo-Spherical Contrastive Divergence [119.28384561517292]
  We propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum learning likelihood of energy-based models. PS-CD avoids the intractable partition function and provides a generalized family of learning objectives.
  arXiv  Detail & Related papers  (2021-11-01T09:17:15Z)
• Causal Graph Discovery from Self and Mutually Exciting Time Series [12.802653884445132]
  We develop a non-asymptotic recovery guarantee and quantifiable uncertainty by solving a linear program. We demonstrate the effectiveness of our approach in recovering highly interpretable causal DAGs over Sepsis Associated Derangements (SADs)
  arXiv  Detail & Related papers  (2021-06-04T16:59:24Z)
• Understanding Overparameterization in Generative Adversarial Networks [56.57403335510056]
  Generative Adversarial Networks (GANs) are used to train non- concave mini-max optimization problems. A theory has shown the importance of the gradient descent (GD) to globally optimal solutions. We show that in an overized GAN with a $1$-layer neural network generator and a linear discriminator, the GDA converges to a global saddle point of the underlying non- concave min-max problem.
  arXiv  Detail & Related papers  (2021-04-12T16:23:37Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)
• On the Benefits of Invariance in Neural Networks [56.362579457990094]
  We show that training with data augmentation leads to better estimates of risk and thereof gradients, and we provide a PAC-Bayes generalization bound for models trained with data augmentation. We also show that compared to data augmentation, feature averaging reduces generalization error when used with convex losses, and tightens PAC-Bayes bounds.
  arXiv  Detail & Related papers  (2020-05-01T02:08:58Z)
• Target-Embedding Autoencoders for Supervised Representation Learning [111.07204912245841]
  This paper analyzes a framework for improving generalization in a purely supervised setting, where the target space is high-dimensional. We motivate and formalize the general framework of target-embedding autoencoders (TEA) for supervised prediction, learning intermediate latent representations jointly optimized to be both predictable from features as well as predictive of targets.
  arXiv  Detail & Related papers  (2020-01-23T02:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.