Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance

• URL: http://arxiv.org/abs/2603.20584v1
• Date: Sat, 21 Mar 2026 00:51:46 GMT
• Title: Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance
• Abstract summary: Diffusion models generate synthetic images through an iterative refinement process.<n>Free Guidance (CFG) and AutoGuidance (AG) alleviate this by extrapolating between the main and inferior signal for stronger generalization.<n>We propose a hybrid instantiation called SGG under the weak-to-strong principle, taking the benefits of both.
• Score: 19.011255034658735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models generate synthetic images through an iterative refinement process. However, the misalignment between the simulation-free objective and the iterative process often causes accumulated gradient error along the sampling trajectory, which leads to unsatisfactory results and a failure to generalize. Guidance techniques like Classifier Free Guidance (CFG) and AutoGuidance (AG) alleviate this by extrapolating between the main and inferior signal for stronger generalization. Despite empirical success, the effective operational regimes of prevalent guidance methods are still under-explored, leading to ambiguity when selecting the appropriate guidance method given a precondition. In this work, we first conduct synthetic comparisons to isolate and demonstrate the effective regime of guidance methods represented by CFG and AG from the perspective of weak-to-strong principle. Based on this, we propose a hybrid instantiation called SGG under the principle, taking the benefits of both. Furthermore, we demonstrate that the W2S principle along with SGG can be migrated into the training objective, improving the generalization ability of unguided diffusion models. We validate our approach with comprehensive experiments. At inference time, evaluations on SD3 and SD3.5 confirm that SGG outperforms existing training-free guidance variants. Training-time experiments on transformer architectures demonstrate the effective migration and performance gains in both conditional and unconditional settings. Code is available at https://github.com/851695e35/SGG.

Related papers

• S-GRPO: Unified Post-Training for Large Vision-Language Models [11.901218794806796]
  We propose Supervised Group Relative Policy Optimization (S-GRPO) for adapting Large Vision-Language Models (LVLMs)<n>S-GRPO integrates the guidance of imitation learning into the multi-trajectory exploration of preference optimization.<n>It gracefully bridges the gap between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL)
  arXiv  Detail & Related papers  (2026-04-17T08:39:07Z)
• One step further with Monte-Carlo sampler to guide diffusion better [11.58337125468802]
  We propose an additional backward denois- ing step and Monte-Carlo sampling (ABMS) to achieve better guided diffu- sion.<n>We conduct experiments across various task settings and data types, mainly includ- ing conditional online handwritten trajectory generation.<n> Experimental results demonstrate that our approach can be effec- tively used with higher order samplers.
  arXiv  Detail & Related papers  (2026-03-04T05:48:02Z)
• GFRRN: Explore the Gaps in Single Image Reflection Removal [23.018215754935753]
  We propose a Gap-Free Reflection Removal Network (GFRRN) for single image reflection removal.<n>In this work, we first adopt the parameter efficient fine-tuning (PEFT) strategy to align the training directions.<n>Then, a label generator is designed to unify the reflection labels for both synthetic and real-world data.<n>Extensive experiments demonstrate the effectiveness of our GFRRN, achieving superior performance against state-of-the-art SIRR methods.
  arXiv  Detail & Related papers  (2026-02-26T07:17:49Z)
• Enhancing Adversarial Transferability by Balancing Exploration and Exploitation with Gradient-Guided Sampling [82.52485740425321]
  Adrial attacks present a critical challenge to deep neural networks' robustness.<n> transferability of adversarial attacks faces a dilemma between Exploitation (maximizing attack potency) and Exploration (enhancing cross-model generalization)
  arXiv  Detail & Related papers  (2025-11-01T05:43:47Z)
• Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models [57.20761595019967]
  We present Normalized Attention Guidance (NAG), an efficient, training-free mechanism that applies extrapolation in attention space with L1-based normalization and refinement.<n>NAG restores effective negative guidance where CFG collapses while maintaining fidelity.<n>NAG generalizes across architectures (UNet, DiT), sampling regimes (few-step, multi-step), and modalities (image, video)
  arXiv  Detail & Related papers  (2025-05-27T13:30:46Z)
• REG: Rectified Gradient Guidance for Conditional Diffusion Models [16.275782069986253]
  We propose rectified gradient guidance (REG) to boost the performance of existing guidance methods.<n>REG provides a better approximation to the optimal solution than prior guidance techniques.<n>In experiments on class-conditional ImageNet and text-to-image generation tasks, REG consistently improves FID and Inception/CLIP scores.
  arXiv  Detail & Related papers  (2025-01-31T03:16:18Z)
• Contrastive CFG: Improving CFG in Diffusion Models by Contrasting Positive and Negative Concepts [55.298031232672734]
  As-Free Guidance (CFG) has proven effective in conditional diffusion model sampling for improved condition alignment. We present a novel method to enhance negative CFG guidance using contrastive loss.
  arXiv  Detail & Related papers  (2024-11-26T03:29:27Z)
• Improving Adversarial Transferability via Intermediate-level Perturbation Decay [79.07074710460012]
  We develop a novel intermediate-level method that crafts adversarial examples within a single stage of optimization. Experimental results show that it outperforms state-of-the-arts by large margins in attacking various victim models.
  arXiv  Detail & Related papers  (2023-04-26T09:49:55Z)
• Orthogonal SVD Covariance Conditioning and Latent Disentanglement [65.67315418971688]
  Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned. We propose Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR) Experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization.
  arXiv  Detail & Related papers  (2022-12-11T20:31:31Z)
• Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality [65.67315418971688]
  Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR) are proposed. Experiments on visual recognition demonstrate that our methods can simultaneously improve the covariance conditioning and generalization.
  arXiv  Detail & Related papers  (2022-07-05T15:39:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.