Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors

• URL: http://arxiv.org/abs/2607.25390v1
• Date: Tue, 28 Jul 2026 07:51:44 GMT
• Title: Noise-Free One-Step LoRA for Task-Driven Image Restoration with Diffusion Priors
• Abstract summary: Degraded images not only reduce visual quality but also impair downstream high-level vision tasks.<n> Task-driven image restoration (TDIR) addresses this issue by jointly optimizing restoration quality and task performance.<n>We show that a deterministic, noise-free one-step forward pass with pretrained diffusion priors can substantially improve TDIR.<n>We also introduce a task-preserving GAN training strategy that improves perceptual quality without sacrificing task performance.
• Score: 54.200377321709546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Degraded images not only reduce visual quality but also impair downstream high-level vision tasks. Task-driven image restoration (TDIR) addresses this issue by jointly optimizing restoration quality and task performance. Recent works show that pretrained diffusion priors benefit TDIR, yet diffusion-based restoration is inherently stochastic, as the sampling process depends on a random noise term, which can undermine task consistency. In this paper, we show that a deterministic, noise-free one-step forward pass with pretrained diffusion priors can substantially improve TDIR, but the benefit critically depends on the adaptation module: LoRA yields consistent gains, whereas ControlNet-style conditioning does not. This enables one-step forwarding that surpasses conventional multi-step diffusion TDIR baselines. Furthermore, we introduce a task-preserving GAN training strategy that improves perceptual quality without sacrificing task performance. Extensive experiments on classification, segmentation, and detection demonstrate consistent gains over prior TDIR methods, and we further validate generalization on real-world degraded images and OCR.

Related papers

• RPL-UIE: Reliable Prior Learning for Underwater Image Enhancement [26.456497000099148]
  We propose RPL-UIE, a framework for reliable prior learning for underwater image enhancement.<n>In the teacher stage, the network learns reliable and complementary spatial priors characterizing appearance and photometric properties from paired degraded and reference images.<n>In the student stage, the network takes only degraded images as input and learns to emulate the teacher's prior extraction capability.
  arXiv  Detail & Related papers  (2026-07-31T14:30:36Z)
• Exploiting Diffusion Prior for Task-driven Image Restoration [52.86792374527662]
  Task-driven image restoration (TDIR) has recently emerged to address performance drops in high-level vision tasks caused by low-quality (LQ) inputs.<n>Previous TDIR methods struggle to handle practical scenarios in which images are degraded by multiple complex factors.<n>We propose EDTR, which effectively harnesses the power of diffusion prior to restore task-relevant details.
  arXiv  Detail & Related papers  (2025-07-30T08:05:49Z)
• LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter [52.93785843453579]
  Blind face restoration from low-quality (LQ) images is a challenging task that requires high-fidelity image reconstruction and the preservation of facial identity.<n>We propose LAFR, a novel codebook-based latent space adapter that aligns the latent distribution of LQ images with that of HQ counterparts.<n>We show that lightweight finetuning of diffusion prior on just 0.9% of FFHQ dataset is sufficient to achieve results comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-29T14:11:16Z)
• UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior [56.35236964617809]
  Image restoration aims to recover content from inputs degraded by various factors, such as adverse weather, blur, and noise.<n>This paper introduces UniRestore, a unified image restoration model that bridges the gap between PIR and TIR.<n>We propose a Complementary Feature Restoration Module (CFRM) to reconstruct degraded encoder features and a Task Feature Adapter (TFA) module to facilitate adaptive feature fusion in the decoder.
  arXiv  Detail & Related papers  (2025-01-22T08:06:48Z)
• Efficient Diffusion as Low Light Enhancer [63.789138528062225]
  Reflectance-Aware Trajectory Refinement (RATR) is a simple yet effective module to refine the teacher trajectory using the reflectance component of images. textbfReflectance-aware textbfDiffusion with textbfDistilled textbfTrajectory (textbfReDDiT) is an efficient and flexible distillation framework tailored for Low-Light Image Enhancement (LLIE)
  arXiv  Detail & Related papers  (2024-10-16T08:07:18Z)
• One-Step Effective Diffusion Network for Real-World Image Super-Resolution [11.326598938246558]
  We propose a one-step effective diffusion network, namely OSEDiff, for the Real-ISR problem. We finetune the pre-trained diffusion network with trainable layers to adapt it to complex image degradations. Our OSEDiff model can efficiently and effectively generate HQ images in just one diffusion step.
  arXiv  Detail & Related papers  (2024-06-12T13:10:31Z)
• Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
  This study proposes a novel and efficient diffusion model for image restoration. Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration. Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
  arXiv  Detail & Related papers  (2024-03-12T05:06:07Z)
• HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models [38.74983301496911]
  Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations. Existing model-based methods have limitations in accurately modeling the complex image characteristics. This paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff)
  arXiv  Detail & Related papers  (2024-02-24T17:15:05Z)
• A Dive into SAM Prior in Image Restoration [40.03648504115027]
  The goal of image restoration (IR) is to restore a high-quality (HQ) image from its degraded low-quality (LQ) observation. We propose a lightweight SAM prior tuning (SPT) unit to integrate semantic priors into existing IR networks. As the only trainable module in our method, the SPT unit has the potential to improve both efficiency and scalability.
  arXiv  Detail & Related papers  (2023-05-23T02:31:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.