UnfoldLDM: Deep Unfolding-based Blind Image Restoration with Latent Diffusion Priors

• URL: http://arxiv.org/abs/2511.18152v1
• Date: Sat, 22 Nov 2025 18:44:01 GMT
• Title: UnfoldLDM: Deep Unfolding-based Blind Image Restoration with Latent Diffusion Priors
• Authors: Chunming He, Rihan Zhang, Zheng Chen, Bowen Yang, CHengyu Fang, Yunlong Lin, Fengyang Xiao, Sina Farsiu,
• Abstract summary: We propose UnfoldLDM to integrate deep unfolding networks with latent diffusion model (LDM) for blind image restoration.<n>In each stage, UnfoldLDM employs a multi-granularity degradation-aware (MGDA) module as the gradient descent step.<n>For the proximal step, we design a degradation-resistant LDM (DR-LDM) to extract compact degradation-invariant priors.<n>This unique combination ensures the final result is degradation-free and visually rich.
• Score: 21.35552789229871
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep unfolding networks (DUNs) combine the interpretability of model-based methods with the learning ability of deep networks, yet remain limited for blind image restoration (BIR). Existing DUNs suffer from: (1) \textbf{Degradation-specific dependency}, as their optimization frameworks are tied to a known degradation model, making them unsuitable for BIR tasks; and (2) \textbf{Over-smoothing bias}, resulting from the direct feeding of gradient descent outputs, dominated by low-frequency content, into the proximal term, suppressing fine textures. To overcome these issues, we propose UnfoldLDM to integrate DUNs with latent diffusion model (LDM) for BIR. In each stage, UnfoldLDM employs a multi-granularity degradation-aware (MGDA) module as the gradient descent step. MGDA models BIR as an unknown degradation estimation problem and estimates both the holistic degradation matrix and its decomposed forms, enabling robust degradation removal. For the proximal step, we design a degradation-resistant LDM (DR-LDM) to extract compact degradation-invariant priors from the MGDA output. Guided by this prior, an over-smoothing correction transformer (OCFormer) explicitly recovers high-frequency components and enhances texture details. This unique combination ensures the final result is degradation-free and visually rich. Experiments show that our UnfoldLDM achieves a leading place on various BIR tasks and benefits downstream tasks. Moreover, our design is compatible with existing DUN-based methods, serving as a plug-and-play framework. Code will be released.

Related papers

• UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration [16.493990086330985]
  UniLDiff is a unified framework enhanced with degradation- and detail-aware mechanisms.<n>We introduce a Degradation-Aware Feature Fusion (DAFF) to dynamically inject low-quality features into each denoising step.<n>We also design a Detail-Aware Expert Module (DAEM) in the decoder to enhance texture and fine-structure recovery.
  arXiv  Detail & Related papers  (2025-07-31T16:02:00Z)
• LD-RPS: Zero-Shot Unified Image Restoration via Latent Diffusion Recurrent Posterior Sampling [38.700993166492495]
  We propose a dataset-free, and unified approach through recurrent posterior sampling utilizing a pretrained latent diffusion model.<n>Our method incorporates the multimodal understanding model to provide sematic priors for the generative model under a task-blind condition.
  arXiv  Detail & Related papers  (2025-07-01T14:25:09Z)
• Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models [19.445391508424667]
  Inverse problems (IPs) involve reconstructing signals from noisy observations.<n>DMs have emerged as a powerful framework for solving IPs, achieving remarkable reconstruction performance.<n>Existing DM-based methods frequently encounter issues such as heavy computational demands and suboptimal convergence.<n>We propose two novel methods, DMILO and DMILO-PGD, to address these challenges.
  arXiv  Detail & Related papers  (2025-05-27T06:49:02Z)
• InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration [106.70903819362402]
  Diffusion priors have been used for blind face restoration (BFR) by fine-tuning diffusion models (DMs) on restoration datasets to recover low-quality images.<n>We propose InterLCM to leverage the latent consistency model (LCM) for its superior semantic consistency and efficiency.<n>InterLCM outperforms existing approaches in both synthetic and real-world datasets while also achieving faster inference speed.
  arXiv  Detail & Related papers  (2025-02-04T10:51:20Z)
• DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration [7.521850476177286]
  We equip diffusion models with the capability to decouple various degradation as a degradation prompt from low-quality (LQ) face images. Our novel restoration scheme, named DR-BFR, guides the denoising of Latent Diffusion Models (LDM) by incorporating Degradation Representation (DR) and content features from LQ images. DR-BFR significantly outperforms state-of-the-art methods quantitatively and qualitatively across various datasets.
  arXiv  Detail & Related papers  (2024-11-15T15:24:42Z)
• DORNet: A Degradation Oriented and Regularized Network for Blind Depth Super-Resolution [48.744290794713905]
  In real-world scenarios, captured depth data often suffer from unconventional and unknown degradation due to sensor limitations and complex imaging environments.<n>We propose the Degradation Oriented and Regularized Network (DORNet), a novel framework designed to adaptively address unknown degradation in real-world scenes.<n>Our approach begins with the development of a self-supervised degradation learning strategy, which models the degradation representations of low-resolution depth data.<n>To facilitate effective RGB-D fusion, we further introduce a degradation-oriented feature transformation module that selectively propagates RGB content into the depth data based on the learned degradation priors.
  arXiv  Detail & Related papers  (2024-10-15T14:53:07Z)
• Suppressing Uncertainties in Degradation Estimation for Blind Super-Resolution [31.89605287039615]
  The problem of blind image super-resolution aims to recover high-resolution (HR) images from low-resolution (LR) images with unknown degradation modes. Most existing methods model the image degradation process using blur kernels. We propose an textbfUncertainty-based degradation representation for blind textbfSuper-textbfResolution framework.
  arXiv  Detail & Related papers  (2024-06-24T08:58:43Z)
• BlindDiff: Empowering Degradation Modelling in Diffusion Models for Blind Image Super-Resolution [52.47005445345593]
  BlindDiff is a DM-based blind SR method to tackle the blind degradation settings in SISR. BlindDiff seamlessly integrates the MAP-based optimization into DMs. Experiments on both synthetic and real-world datasets show that BlindDiff achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-15T11:21:34Z)
• Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model [59.08821399652483]
  Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion model (DM)-based methods have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. We propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task. Reti-Diff comprises two key components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RG
  arXiv  Detail & Related papers  (2023-11-20T09:55:06Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• Meta-Learning based Degradation Representation for Blind Super-Resolution [54.93926549648434]
  We propose a Meta-Learning based Region Degradation Aware SR Network (MRDA) We use the MRDA to rapidly adapt to the specific complex degradation after several iterations and extract implicit degradation information. A teacher network MRDA$_T$ is designed to further utilize the degradation information extracted by MLN for SR.
  arXiv  Detail & Related papers  (2022-07-28T09:03:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.