Exploiting Diffusion Prior for Task-driven Image Restoration

• URL: http://arxiv.org/abs/2507.22459v2
• Date: Tue, 02 Sep 2025 02:46:49 GMT
• Title: Exploiting Diffusion Prior for Task-driven Image Restoration
• Authors: Jaeha Kim, Junghun Oh, Kyoung Mu Lee,
• Abstract summary: Task-driven image restoration (TDIR) has recently emerged to address performance drops in high-level vision tasks caused by low-quality (LQ) inputs.<n>Previous TDIR methods struggle to handle practical scenarios in which images are degraded by multiple complex factors.<n>We propose EDTR, which effectively harnesses the power of diffusion prior to restore task-relevant details.
• Score: 52.86792374527662
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Task-driven image restoration (TDIR) has recently emerged to address performance drops in high-level vision tasks caused by low-quality (LQ) inputs. Previous TDIR methods struggle to handle practical scenarios in which images are degraded by multiple complex factors, leaving minimal clues for restoration. This motivates us to leverage the diffusion prior, one of the most powerful natural image priors. However, while the diffusion prior can help generate visually plausible results, using it to restore task-relevant details remains challenging, even when combined with recent TDIR methods. To address this, we propose EDTR, which effectively harnesses the power of diffusion prior to restore task-relevant details. Specifically, we propose directly leveraging useful clues from LQ images in the diffusion process by generating from pixel-error-based pre-restored LQ images with mild noise added. Moreover, we employ a small number of denoising steps to prevent the generation of redundant details that dilute crucial task-related information. We demonstrate that our method effectively utilizes diffusion prior for TDIR, significantly enhancing task performance and visual quality across diverse tasks with multiple complex degradations.

Related papers

• Extreme Blind Image Restoration via Prompt-Conditioned Information Bottleneck [47.027290803102666]
  Blind Image Restoration (BIR) methods have achieved remarkable success but falter when faced with Extreme Blind Image Restoration (EBIR)<n>We propose a novel framework that decomposes the intractable ELQ-to-HQ restoration process.
  arXiv  Detail & Related papers  (2025-10-01T10:13:27Z)
• Decouple to Reconstruct: High Quality UHD Restoration via Active Feature Disentanglement and Reversible Fusion [77.08942160610478]
  Ultra-high-definition (UHD) image restoration often faces computational bottlenecks and information loss due to its extremely high resolution.<n>We propose a Controlled Differential Disentangled VAE that discards easily recoverable background information while encoding more difficult-to-recover degraded information into latent space.<n>Our method effectively alleviates the information loss problem in VAE models while ensuring computational efficiency, significantly improving the quality of UHD image restoration, and achieves state-of-the-art results in six UHD restoration tasks with only 1M parameters.
  arXiv  Detail & Related papers  (2025-03-17T02:55:18Z)
• UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior [56.35236964617809]
  Image restoration aims to recover content from inputs degraded by various factors, such as adverse weather, blur, and noise.<n>This paper introduces UniRestore, a unified image restoration model that bridges the gap between PIR and TIR.<n>We propose a Complementary Feature Restoration Module (CFRM) to reconstruct degraded encoder features and a Task Feature Adapter (TFA) module to facilitate adaptive feature fusion in the decoder.
  arXiv  Detail & Related papers  (2025-01-22T08:06:48Z)
• Prior Knowledge Distillation Network for Face Super-Resolution [25.188937155619886]
  The purpose of face super-resolution (FSR) is to reconstruct high-resolution (HR) face images from low-resolution (LR) inputs. We propose a prior knowledge distillation network (PKDN) for FSR, which involves transferring prior information from the teacher network to the student network.
  arXiv  Detail & Related papers  (2024-09-22T09:58:20Z)
• One Step Diffusion-based Super-Resolution with Time-Aware Distillation [60.262651082672235]
  Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. Recent techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. We propose a time-aware diffusion distillation method, named TAD-SR, to accomplish effective and efficient image super-resolution.
  arXiv  Detail & Related papers  (2024-08-14T11:47:22Z)
• One-Step Effective Diffusion Network for Real-World Image Super-Resolution [11.326598938246558]
  We propose a one-step effective diffusion network, namely OSEDiff, for the Real-ISR problem. We finetune the pre-trained diffusion network with trainable layers to adapt it to complex image degradations. Our OSEDiff model can efficiently and effectively generate HQ images in just one diffusion step.
  arXiv  Detail & Related papers  (2024-06-12T13:10:31Z)
• DiffuseRAW: End-to-End Generative RAW Image Processing for Low-Light Images [5.439020425819001]
  We develop a new generative ISP that relies on fine-tuning latent diffusion models on RAW images. We evaluate our approach on popular end-to-end low-light datasets for which we see promising results.
  arXiv  Detail & Related papers  (2023-12-13T03:39:05Z)
• Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model [59.08821399652483]
  Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion model (DM)-based methods have shown promising performance but are often burdened by heavy computational demands and pixel misalignment issues when predicting the image-level distribution. We propose to leverage DM within a compact latent space to generate concise guidance priors and introduce a novel solution called Reti-Diff for the IDIR task. Reti-Diff comprises two key components: the Retinex-based latent DM (RLDM) and the Retinex-guided transformer (RG
  arXiv  Detail & Related papers  (2023-11-20T09:55:06Z)
• TAPE: Task-Agnostic Prior Embedding for Image Restoration [194.61997784161218]
  We propose a novel approach that embeds a task-agnostic prior into a transformer. Our approach, named Task-Agnostic Prior Embedding (TAPE), consists of three stages, namely, task-agnostic pre-training, task-agnostic fine-tuning, and task-specific fine-tuning. Experiments on various types of degradation validate the effectiveness of TAPE.
  arXiv  Detail & Related papers  (2022-03-11T16:52:47Z)
• Deep-Masking Generative Network: A Unified Framework for Background Restoration from Superimposed Images [36.7646332887842]
  We present the Deep-Masking Generative Network (DMGN), which is a unified framework for background restoration from superimposed images. A coarse background image and a noise image are first generated in parallel, then the noise image is further leveraged to refine the background image. Our experiments show that our DMGN consistently outperforms state-of-the-art methods specifically designed for each single task.
  arXiv  Detail & Related papers  (2020-10-09T01:47:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.