Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation

• URL: http://arxiv.org/abs/2602.22570v1
• Date: Thu, 26 Feb 2026 03:24:11 GMT
• Title: Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generation
• Abstract summary: Can emerging diffusion guidance methods achieve solid and significant improvements?<n>We reveal a critical evaluation pitfall that common human preference models exhibit a strong bias towards large guidance scales.<n>Second, we introduce a novel guidance-aware evaluation (GA-Eval) framework that employs effective guidance scale calibration.<n>Third, motivated by the evaluation pitfall, we design Transcendent Diffusion Guidance (TDG) method that can significantly improve human preference scores in the conventional evaluation framework but actually does not work in practice.
• Score: 27.855521267795208
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Classifier-free guidance (CFG) has helped diffusion models achieve great conditional generation in various fields. Recently, more diffusion guidance methods have emerged with improved generation quality and human preference. However, can these emerging diffusion guidance methods really achieve solid and significant improvements? In this paper, we rethink recent progress on diffusion guidance. Our work mainly consists of four contributions. First, we reveal a critical evaluation pitfall that common human preference models exhibit a strong bias towards large guidance scales. Simply increasing the CFG scale can easily improve quantitative evaluation scores due to strong semantic alignment, even if image quality is severely damaged (e.g., oversaturation and artifacts). Second, we introduce a novel guidance-aware evaluation (GA-Eval) framework that employs effective guidance scale calibration to enable fair comparison between current guidance methods and CFG by identifying the effects orthogonal and parallel to CFG effects. Third, motivated by the evaluation pitfall, we design Transcendent Diffusion Guidance (TDG) method that can significantly improve human preference scores in the conventional evaluation framework but actually does not work in practice. Fourth, in extensive experiments, we empirically evaluate recent eight diffusion guidance methods within the conventional evaluation framework and the proposed GA-Eval framework. Notably, simply increasing the CFG scales can compete with most studied diffusion guidance methods, while all methods suffer severely from winning rate degradation over standard CFG. Our work would strongly motivate the community to rethink the evaluation paradigm and future directions of this field.

Related papers

• Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models [0.2730969268472861]
  Inference-time quality-enhancement methods are widely adopted to improve diffusion models.<n>We re-evaluate eight such methods on two open-weight transformers under a fixed per-model protocol.<n>No method consistently improves on CFG across the measured criteria.
  arXiv  Detail & Related papers  (2026-08-17T16:40:54Z)
• Echoes in Filter Bubble: Diagnosing and Curing Popularity Bias in Generative Recommenders [65.5347571901244]
  This study focuses on the optimization of generative framework and the item tokenization based on semantic index.<n>Based on theoretical analyses, we identify that the severe popularity bias emerges from the confluence of a token-level optimization flaw and the undifferentiated property of item tokenization.<n>This study develops a novel generative recommender system, called Ghost, by designing the asymmetric unlikelihood optimization and the skeleton-founded tokenization.
  arXiv  Detail & Related papers  (2026-05-16T06:02:40Z)
• Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance [19.011255034658735]
  Diffusion models generate synthetic images through an iterative refinement process.<n>Free Guidance (CFG) and AutoGuidance (AG) alleviate this by extrapolating between the main and inferior signal for stronger generalization.<n>We propose a hybrid instantiation called SGG under the weak-to-strong principle, taking the benefits of both.
  arXiv  Detail & Related papers  (2026-03-21T00:51:46Z)
• Towards a Golden Classifier-Free Guidance Path via Foresight Fixed Point Iterations [12.366757123129402]
  We propose a unified perspective that reframes conditional guidance as fixed point iterations.<n>We introduce Foresight Guidance (FSG), which prioritizes solving longer-interval subproblems in early diffusion stages.<n>Our work offers novel perspectives for conditional guidance and unlocks the potential of adaptive design.
  arXiv  Detail & Related papers  (2025-10-24T14:39:07Z)
• How Much To Guide: Revisiting Adaptive Guidance in Classifier-Free Guidance Text-to-Vision Diffusion Models [57.42800112251644]
  We propose Step AG, which is a simple, universally applicable adaptive guidance strategy.<n>Our evaluations focus on both image quality and image-text alignment.
  arXiv  Detail & Related papers  (2025-06-10T02:09:48Z)
• A Meaningful Perturbation Metric for Evaluating Explainability Methods [55.09730499143998]
  We introduce a novel approach, which harnesses image generation models to perform targeted perturbation.<n> Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity.<n>This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results.
  arXiv  Detail & Related papers  (2025-04-09T11:46:41Z)
• Domain Guidance: A Simple Transfer Approach for a Pre-trained Diffusion Model [62.11981915549919]
  Domain Guidance is a transfer approach that leverages pre-trained knowledge to guide the sampling process toward the target domain.<n>We demonstrate its substantial effectiveness across various transfer benchmarks, achieving over a 19.6% improvement in FID and a 23.4% improvement in FD$_textDINOv2$ compared to standard fine-tuning.
  arXiv  Detail & Related papers  (2025-04-02T09:07:55Z)
• REG: Rectified Gradient Guidance for Conditional Diffusion Models [16.275782069986253]
  We propose rectified gradient guidance (REG) to boost the performance of existing guidance methods.<n>REG provides a better approximation to the optimal solution than prior guidance techniques.<n>In experiments on class-conditional ImageNet and text-to-image generation tasks, REG consistently improves FID and Inception/CLIP scores.
  arXiv  Detail & Related papers  (2025-01-31T03:16:18Z)
• Nested Annealed Training Scheme for Generative Adversarial Networks [54.70743279423088]
  This paper focuses on a rigorous mathematical theoretical framework: the composite-functional-gradient GAN (CFG)<n>We reveal the theoretical connection between the CFG model and score-based models.<n>We find that the training objective of the CFG discriminator is equivalent to finding an optimal D(x)
  arXiv  Detail & Related papers  (2025-01-20T07:44:09Z)
• Contrastive CFG: Improving CFG in Diffusion Models by Contrasting Positive and Negative Concepts [55.298031232672734]
  As-Free Guidance (CFG) has proven effective in conditional diffusion model sampling for improved condition alignment. We present a novel method to enhance negative CFG guidance using contrastive loss.
  arXiv  Detail & Related papers  (2024-11-26T03:29:27Z)
• Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models [44.58960475893552]
  "Adaptive Guidance" (AG) is an efficient variant of computation-Free Guidance (CFG) AG preserves CFG's image quality while reducing by 25%. " LinearAG" offers even cheaper inference at the cost of deviating from the baseline model.
  arXiv  Detail & Related papers  (2023-12-19T17:08:48Z)
• Exploring the Optimization Objective of One-Class Classification for Anomaly Detection [2.9266769103356305]
  One-class classification (OCC) is a longstanding method for anomaly detection. In this work, we conduct a thorough investigation into the optimization objective of OCC. We unveil a key insights: any space with the suitable norm can serve as an equivalent substitute for the hypersphere center. This novel insight sparks a simple and data-agnostic deep one-class classification method.
  arXiv  Detail & Related papers  (2023-08-23T03:46:04Z)
• Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems [84.3996727203154]
  We show that a popular choice of contrastive loss is equivalent to reducing the exposure bias via inverse propensity weighting. We further improve upon CLRec and propose Multi-CLRec, for accurate multi-intention aware bias reduction. Our methods have been successfully deployed in Taobao, where at least four-month online A/B tests and offline analyses demonstrate its substantial improvements.
  arXiv  Detail & Related papers  (2020-05-20T08:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.