AdaptIR: Parameter Efficient Multi-task Adaptation for Pre-trained Image Restoration Models

• URL: http://arxiv.org/abs/2312.08881v1
• Date: Tue, 12 Dec 2023 14:27:59 GMT
• Title: AdaptIR: Parameter Efficient Multi-task Adaptation for Pre-trained Image Restoration Models
• Authors: Hang Guo, Tao Dai, Yuanchao Bai, Bin Chen, Shu-Tao Xia, Zexuan Zhu
• Abstract summary: We propose AdaptIR, a novel parameter efficient transfer learning method for adapting pre-trained restoration models. Experiments demonstrate that the proposed method can achieve comparable or even better performance than full fine-tuning, while only using 0.6%.
• Score: 58.10797482129863
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Pre-training has shown promising results on various image restoration tasks, which is usually followed by full fine-tuning for each specific downstream task (e.g., image denoising). However, such full fine-tuning usually suffers from the problems of heavy computational cost in practice, due to the massive parameters of pre-trained restoration models, thus limiting its real-world applications. Recently, Parameter Efficient Transfer Learning (PETL) offers an efficient alternative solution to full fine-tuning, yet still faces great challenges for pre-trained image restoration models, due to the diversity of different degradations. To address these issues, we propose AdaptIR, a novel parameter efficient transfer learning method for adapting pre-trained restoration models. Specifically, the proposed method consists of a multi-branch inception structure to orthogonally capture local spatial, global spatial, and channel interactions. In this way, it allows powerful representations under a very low parameter budget. Extensive experiments demonstrate that the proposed method can achieve comparable or even better performance than full fine-tuning, while only using 0.6% parameters. Code is available at https://github.com/csguoh/AdaptIR.

Related papers

• UniRes: Universal Image Restoration for Complex Degradations [53.74404005987783]
  Real-world image restoration is hampered by diverse degradations stemming from varying capture conditions, capture devices and post-processing pipelines.<n>A simple yet flexible diffusionbased framework, named UniRes, is proposed to address such degradations in an end-to-end manner.<n>Our proposed method is evaluated on both complex-degradation and single-degradation image restoration datasets.
  arXiv  Detail & Related papers  (2025-06-05T21:25:39Z)
• DPMambaIR:All-in-One Image Restoration via Degradation-Aware Prompt State Space Model [36.979833523678614]
  All-in-One image restoration aims to address multiple image degradation problems. Existing approaches rely on Degradation-specific models or coarse-grained degradation prompts to guide image restoration. We propose DPMambaIR, a novel All-in-One image restoration framework.
  arXiv  Detail & Related papers  (2025-04-24T16:46:32Z)
• Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration [109.38288333994407]
  Contrastive Prompt Learning (CPL) is a novel framework that fundamentally enhances prompt-task alignment. Our framework establishes new state-of-the-art performance while maintaining parameter efficiency, offering a principled solution for unified image restoration.
  arXiv  Detail & Related papers  (2025-04-14T08:24:57Z)
• UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity [79.90839080916913]
  We present our UniRestorer with improved restoration performance. Specifically, we perform hierarchical clustering on degradation space, and train a multi-granularity mixture-of-experts (MoE) restoration model. In contrast to existing degradation-agnostic and -aware methods, UniRestorer can leverage degradation estimation to benefit degradation specific restoration.
  arXiv  Detail & Related papers  (2024-12-28T14:09:08Z)
• Adaptive Blind All-in-One Image Restoration [15.726917603679716]
  Blind all-in-one image restoration models aim to recover a high-quality image from an input degraded with unknown distortions. We introduce ABAIR, a simple yet effective adaptive blind all-in-one restoration model that handles multiple degradations and generalizes well to unseen distortions. Our model not only surpasses state-of-the-art performance on five- and three-task IR setups but also demonstrates superior generalization to unseen degradations and composite distortions.
  arXiv  Detail & Related papers  (2024-11-27T14:58:08Z)
• Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding [67.57487747508179]
  Multiple-in-one image restoration (IR) has made significant progress, aiming to handle all types of single degraded image restoration with a single model. In this paper, we propose a novel multiple-in-one IR model that can effectively restore images with both single and mixed degradations.
  arXiv  Detail & Related papers  (2024-11-25T09:26:34Z)
• Chain-of-Restoration: Multi-Task Image Restoration Models are Zero-Shot Step-by-Step Universal Image Restorers [53.298698981438]
  We propose Universal Image Restoration (UIR), a new task setting that requires models to be trained on a set of degradation bases and then remove any degradation that these bases can potentially compose in a zero-shot manner. Inspired by the Chain-of-Thought which prompts LLMs to address problems step-by-step, we propose the Chain-of-Restoration (CoR) CoR instructs models to step-by-step remove unknown composite degradations.
  arXiv  Detail & Related papers  (2024-10-11T10:21:42Z)
• UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation [50.27688690379488]
  Existing unified methods treat multi-degradation image restoration as a multi-task learning problem. We propose a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks.
  arXiv  Detail & Related papers  (2024-09-30T11:16:56Z)
• Taming Generative Diffusion Prior for Universal Blind Image Restoration [4.106012295148947]
  BIR-D is able to fulfill multi-guidance blind image restoration. It can also restore images that undergo multiple and complicated degradations, demonstrating the practical applications.
  arXiv  Detail & Related papers  (2024-08-21T02:19:54Z)
• HAIR: Hypernetworks-based All-in-One Image Restoration [46.681872835394095]
  Hair is a Hypernetworks-based All-in-One Image Restoration plug-and-play method. It generates parameters based on the input image and thus makes the model to adapt to specific degradation dynamically. It can significantly improve the performance of existing image restoration models in a plug-and-play manner, both in single-task and All-in-One settings.
  arXiv  Detail & Related papers  (2024-08-15T11:34:33Z)
• Any Image Restoration with Efficient Automatic Degradation Adaptation [132.81912195537433]
  We propose a unified manner to achieve joint embedding by leveraging the inherent similarities across various degradations for efficient and comprehensive restoration. Our network sets new SOTA records while reducing model complexity by approximately -82% in trainable parameters and -85% in FLOPs.
  arXiv  Detail & Related papers  (2024-07-18T10:26:53Z)
• PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance [65.5618804029422]
  Previous works have achieved noteworthy success by limiting the solution space using explicit degradation models. We propose PGDiff by introducing partial guidance, a fresh perspective that is more adaptable to real-world degradations. Our method not only outperforms existing diffusion-prior-based approaches but also competes favorably with task-specific models.
  arXiv  Detail & Related papers  (2023-09-19T17:51:33Z)
• One Size Fits All: Hypernetwork for Tunable Image Restoration [5.33024001730262]
  We introduce a novel approach for tunable image restoration that achieves the accuracy of multiple models, each optimized for a different level of degradation. Our model can be optimized to restore as many degradation levels as required with a constant number of parameters and for various image restoration tasks.
  arXiv  Detail & Related papers  (2022-06-13T08:33:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.