Enhancing Diffusion Model Stability for Image Restoration via Gradient Management

• URL: http://arxiv.org/abs/2507.06656v2
• Date: Wed, 03 Sep 2025 05:42:46 GMT
• Title: Enhancing Diffusion Model Stability for Image Restoration via Gradient Management
• Abstract summary: We analyze the underlying gradient dynamics of these components and identify significant instabilities.<n>We show that these instabilities disrupt the generative process and compromise restoration performance.<n>We propose Stabilized Progressive Gradient Diffusion, a novel gradient management technique.
• Score: 25.215396311196713
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models have shown remarkable promise for image restoration by leveraging powerful priors. Prominent methods typically frame the restoration problem within a Bayesian inference framework, which iteratively combines a denoising step with a likelihood guidance step. However, the interactions between these two components in the generation process remain underexplored. In this paper, we analyze the underlying gradient dynamics of these components and identify significant instabilities. Specifically, we demonstrate conflicts between the prior and likelihood gradient directions, alongside temporal fluctuations in the likelihood gradient itself. We show that these instabilities disrupt the generative process and compromise restoration performance. To address these issues, we propose Stabilized Progressive Gradient Diffusion (SPGD), a novel gradient management technique. SPGD integrates two synergistic components: (1) a progressive likelihood warm-up strategy to mitigate gradient conflicts; and (2) adaptive directional momentum (ADM) smoothing to reduce fluctuations in the likelihood gradient. Extensive experiments across diverse restoration tasks demonstrate that SPGD significantly enhances generation stability, leading to state-of-the-art performance in quantitative metrics and visually superior results. Code is available at https://github.com/74587887/SPGD.

Related papers

• Heterogeneous Knowledge Distillation via Geometry Decoupling and Momentum-Aware Gradient Regulation [3.619179880246355]
  SPOFA is a framework built upon a novel Feature and Gradient Dual Stabilization mechanism.<n>MEMA actively evaluates instantaneous gradient conflicts and adaptively penalizes harmful distillation signals.<n>Experiments on two mainstream benchmarks demonstrate that SPOFA achieves state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2026-06-23T13:23:58Z)
• Focus, Align, and Sustain: Counteracting Gradient Dilution in Incremental Object Detection [34.23489857539403]
  We propose a unified framework thatFocuses, Aligns, and Sustains gradient flow throughout incremental learning.<n>Experiments show that FAS restores robust optimization dynamics and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-13T11:20:16Z)
• HarmoGS: Robust 3D Gaussian Splatting in the Wild via Conflict-Aware Gradient Harmonization [82.42818558196615]
  In-the-wild 3D Gaussian Splatting remains challenging due to transient distractors and illumination-induced cross-view appearance inconsistencies.<n>We propose a conflict-aware 3DGS framework that addresses this problem from both image-space supervision and gradient-level optimization.<n>Our method achieves state-of-the-art rendering quality under complex transient distractors and cross-view inconsistencies.
  arXiv  Detail & Related papers  (2026-05-13T06:47:09Z)
• SURGE: Surrogate Gradient Adaptation in Binary Neural Networks [41.424349870612716]
  We propose SURrogate GradiEnt Adaptation (SURGE), a learnable gradient compensation framework with theoretical grounding.<n> SURGE mitigates gradient mismatch through auxiliary backpropagation.<n> Experiments on image classification, object detection, and language understanding tasks demonstrate that SURGE performs best over state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-09T09:52:38Z)
• Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction [45.25461515976432]
  Plug-and-Play diffusion prior (DP) frameworks have emerged as a powerful paradigm for imaging reconstruction.<n>We present a novel approach to resolving bias-hallucination trade-off, achieving state-of-the-art gradients with significantly accelerated convergence.
  arXiv  Detail & Related papers  (2026-02-26T16:58:43Z)
• All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradations [102.94052335735326]
  All-in-one image restoration aims to recover clean images from diverse unknown degradations using a single model.<n>Existing approaches primarily focus on frame-wise degradation variation, overlooking the temporal continuity that naturally exists in real-world degradation processes.<n>We introduce the Smoothly Evolving Unknown Degradations (SEUD) scenario, where both the active degradation set and degradation intensity change continuously over time.
  arXiv  Detail & Related papers  (2026-01-02T02:20:57Z)
• Robust Posterior Diffusion-based Sampling via Adaptive Guidance Scale [39.27744518020771]
  We propose an adaptive likelihood step-size strategy to guide the diffusion process for inverse-problem formulations.<n>The resulting approach, Adaptive Posterior diffusion Sampling (AdaPS), is hyper-free and improves reconstruction quality across diverse imaging tasks.
  arXiv  Detail & Related papers  (2025-11-23T14:37:59Z)
• Boosting Fidelity for Pre-Trained-Diffusion-Based Low-Light Image Enhancement via Condition Refinement [63.54516423266521]
  Pre-Trained Diffusion-Based (PTDB) methods often sacrifice content fidelity to attain higher perceptual realism.<n>We propose a novel optimization strategy for conditioning in pre-trained diffusion models, enhancing fidelity while preserving realism and aesthetics.<n>Our approach is plug-and-play, seamlessly integrating into existing diffusion networks to provide more effective control.
  arXiv  Detail & Related papers  (2025-10-20T02:40:06Z)
• DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing [73.12011187146481]
  Inversion within Diffusion models aims to recover the latent noise representation for a real or generated image.<n>Most inversion approaches suffer from an intrinsic trade-off between reconstruction accuracy and editing flexibility.<n>We introduce Dual-Conditional Inversion (DCI), a novel framework that jointly conditions on the source prompt and reference image.
  arXiv  Detail & Related papers  (2025-06-03T07:46:44Z)
• Harmonizing Intra-coherence and Inter-divergence in Ensemble Attacks for Adversarial Transferability [29.69932132116216]
  We propose a novel method Harmonized Ensemble for Adversarial Transferability (HEAT)<n>Heated Ensemble for Adversarial Transferability introduces domain generalization into adversarial example generation for the first time.<n> Experimental results demonstrate that HEAT significantly outperforms existing methods across various datasets and settings.
  arXiv  Detail & Related papers  (2025-05-02T10:17:33Z)
• Aligning Foundation Model Priors and Diffusion-Based Hand Interactions for Occlusion-Resistant Two-Hand Reconstruction [50.952228546326516]
  Two-hand reconstruction from monocular images faces persistent challenges due to complex and dynamic hand postures and occlusions.<n>Existing approaches struggle with such alignment issues, often resulting in misalignment and penetration artifacts.<n>We propose a novel framework that attempts to precisely align hand poses and interactions by integrating foundation model-driven 2D priors with diffusion-based interaction refinement.
  arXiv  Detail & Related papers  (2025-03-22T14:42:27Z)
• FreeBlend: Advancing Concept Blending with Staged Feedback-Driven Interpolation Diffusion [12.699651619563188]
  Concept blending is a promising yet underexplored area in generative models.<n>We introduce FreeBlend, an effective, training-free framework designed to address these challenges.
  arXiv  Detail & Related papers  (2025-02-08T15:25:03Z)
• Mjolnir: Breaking the Shield of Perturbation-Protected Gradients via Adaptive Diffusion [13.764770382623812]
  We present the first attempt to break the shield of gradient perturbation protection in Federated Learning.<n>We introduce Mjolnir, a perturbation-resilient gradient leakage attack.<n>Mjolnir is capable of removing perturbations from gradients without requiring additional access to the original model structure or external data.
  arXiv  Detail & Related papers  (2024-07-07T07:06:49Z)
• Adaptive Adversarial Cross-Entropy Loss for Sharpness-Aware Minimization [2.8775022881551666]
  Sharpness-Aware Minimization (SAM) was proposed to enhance model generalization. SAM consists of two main steps, the weight perturbation step and the weight updating step. We propose the Adaptive Adversarial Cross-Entropy (AACE) loss function to replace standard cross-entropy loss for SAM's perturbation.
  arXiv  Detail & Related papers  (2024-06-20T14:00:01Z)
• Boosting Visual Recognition in Real-world Degradations via Unsupervised Feature Enhancement Module with Deep Channel Prior [22.323789227447755]
  Fog, low-light, and motion blur degrade image quality and pose threats to the safety of autonomous driving. This work proposes a novel Deep Channel Prior (DCP) for degraded visual recognition. Based on this, a novel plug-and-play Unsupervised Feature Enhancement Module (UFEM) is proposed to achieve unsupervised feature correction.
  arXiv  Detail & Related papers  (2024-04-02T07:16:56Z)
• Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model [59.08821399652483]
  Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion model (DM)-based methods have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. We propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task. Reti-Diff comprises two key components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RG
  arXiv  Detail & Related papers  (2023-11-20T09:55:06Z)
• PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance [65.5618804029422]
  Previous works have achieved noteworthy success by limiting the solution space using explicit degradation models. We propose PGDiff by introducing partial guidance, a fresh perspective that is more adaptable to real-world degradations. Our method not only outperforms existing diffusion-prior-based approaches but also competes favorably with task-specific models.
  arXiv  Detail & Related papers  (2023-09-19T17:51:33Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• DR2: Diffusion-based Robust Degradation Remover for Blind Face Restoration [66.01846902242355]
  Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training. It is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. We propose Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-invariant prediction, then employ an enhancement module to restore the coarse prediction to a high-quality image.
  arXiv  Detail & Related papers  (2023-03-13T06:05:18Z)
• Cogradient Descent for Bilinear Optimization [124.45816011848096]
  We introduce a Cogradient Descent algorithm (CoGD) to address the bilinear problem. We solve one variable by considering its coupling relationship with the other, leading to a synchronous gradient descent. Our algorithm is applied to solve problems with one variable under the sparsity constraint.
  arXiv  Detail & Related papers  (2020-06-16T13:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.