UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity

• URL: http://arxiv.org/abs/2412.20157v1
• Date: Sat, 28 Dec 2024 14:09:08 GMT
• Title: UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity
• Authors: Jingbo Lin, Zhilu Zhang, Wenbo Li, Renjing Pei, Hang Xu, Hongzhi Zhang, Wangmeng Zuo,
• Abstract summary: We present our UniRestorer with improved restoration performance. Specifically, we perform hierarchical clustering on degradation space, and train a multi-granularity mixture-of-experts (MoE) restoration model. In contrast to existing degradation-agnostic and -aware methods, UniRestorer can leverage degradation estimation to benefit degradationspecific restoration.
• Score: 79.90839080916913
• License:
• Abstract: Recently, considerable progress has been made in allin-one image restoration. Generally, existing methods can be degradation-agnostic or degradation-aware. However, the former are limited in leveraging degradation-specific restoration, and the latter suffer from the inevitable error in degradation estimation. Consequently, the performance of existing methods has a large gap compared to specific single-task models. In this work, we make a step forward in this topic, and present our UniRestorer with improved restoration performance. Specifically, we perform hierarchical clustering on degradation space, and train a multi-granularity mixture-of-experts (MoE) restoration model. Then, UniRestorer adopts both degradation and granularity estimation to adaptively select an appropriate expert for image restoration. In contrast to existing degradation-agnostic and -aware methods, UniRestorer can leverage degradation estimation to benefit degradationspecific restoration, and use granularity estimation to make the model robust to degradation estimation error. Experimental results show that our UniRestorer outperforms stateof-the-art all-in-one methods by a large margin, and is promising in closing the performance gap to specific single task models. The code and pre-trained models will be publicly available at https://github.com/mrluin/UniRestorer.

Related papers

• Mixed Degradation Image Restoration via Local Dynamic Optimization and Conditional Embedding [67.57487747508179]
  Multiple-in-one image restoration (IR) has made significant progress, aiming to handle all types of single degraded image restoration with a single model. In this paper, we propose a novel multiple-in-one IR model that can effectively restore images with both single and mixed degradations.
  arXiv  Detail & Related papers  (2024-11-25T09:26:34Z)
• UIR-LoRA: Achieving Universal Image Restoration through Multiple Low-Rank Adaptation [50.27688690379488]
  Existing unified methods treat multi-degradation image restoration as a multi-task learning problem. We propose a universal image restoration framework based on multiple low-rank adapters (LoRA) from multi-domain transfer learning. Our framework leverages the pre-trained generative model as the shared component for multi-degradation restoration and transfers it to specific degradation image restoration tasks.
  arXiv  Detail & Related papers  (2024-09-30T11:16:56Z)
• Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration [19.87693298262894]
  We propose Diff-Restorer, a universal image restoration method based on the diffusion model. We utilize the pre-trained visual language model to extract visual prompts from degraded images. We also design a Degradation-aware Decoder to perform structural correction and convert the latent code to the pixel domain.
  arXiv  Detail & Related papers  (2024-07-04T05:01:10Z)
• Efficient Degradation-aware Any Image Restoration [83.92870105933679]
  We propose textitDaAIR, an efficient All-in-One image restorer employing a Degradation-aware Learner (DaLe) in the low-rank regime. By dynamically allocating model capacity to input degradations, we realize an efficient restorer integrating holistic and specific learning.
  arXiv  Detail & Related papers  (2024-05-24T11:53:27Z)
• InstructIR: High-Quality Image Restoration Following Human Instructions [61.1546287323136]
  We present the first approach that uses human-written instructions to guide the image restoration model. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks.
  arXiv  Detail & Related papers  (2024-01-29T18:53:33Z)
• PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance [65.5618804029422]
  Previous works have achieved noteworthy success by limiting the solution space using explicit degradation models. We propose PGDiff by introducing partial guidance, a fresh perspective that is more adaptable to real-world degradations. Our method not only outperforms existing diffusion-prior-based approaches but also competes favorably with task-specific models.
  arXiv  Detail & Related papers  (2023-09-19T17:51:33Z)
• All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation [47.00239809958627]
  We propose a novel All-in-one Multi-degradation Image Restoration Network (AMIRNet) AMIRNet learns a degradation representation for unknown degraded images by progressively constructing a tree structure through clustering. This tree-structured representation explicitly reflects the consistency and discrepancy of various distortions, providing a specific clue for image restoration.
  arXiv  Detail & Related papers  (2023-08-06T04:51:41Z)
• Robust Unsupervised StyleGAN Image Restoration [5.33024001730262]
  GAN-based image restoration inverts the generative process to repair images corrupted by known degradations. We make StyleGAN image restoration robust, working across a wide range of degradation levels. Our proposed approach relies on a 3-phase progressive latent space extension and a conservative robustness.
  arXiv  Detail & Related papers  (2023-02-13T22:45:54Z)
• Perceptual Image Restoration with High-Quality Priori and Degradation Learning [28.93489249639681]
  We show that our model performs well in measuring the similarity between restored and degraded images. Our simultaneous restoration and enhancement framework generalizes well to real-world complicated degradation types.
  arXiv  Detail & Related papers  (2021-03-04T13:19:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.