Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration

• URL: http://arxiv.org/abs/2408.15994v1
• Date: Wed, 28 Aug 2024 17:58:54 GMT
• Title: Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration
• Authors: Xu Zhang, Jiaqi Ma, Guoli Wang, Qian Zhang, Huan Zhang, Lefei Zhang,
• Abstract summary: Perceive-IR is an all-in-one image restorer designed to achieve fine-grained quality control. In the prompt learning stage, we leverage prompt learning to acquire a fine-grained quality perceiver capable of distinguishing three-tier quality levels. For the restoration stage, a semantic guidance module and compact feature extraction are proposed to further promote the restoration process.
• Score: 33.163161549726446
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The limitations of task-specific and general image restoration methods for specific degradation have prompted the development of all-in-one image restoration techniques. However, the diversity of patterns among multiple degradation, along with the significant uncertainties in mapping between degraded images of different severities and their corresponding undistorted versions, pose significant challenges to the all-in-one restoration tasks. To address these challenges, we propose Perceive-IR, an all-in-one image restorer designed to achieve fine-grained quality control that enables restored images to more closely resemble their undistorted counterparts, regardless of the type or severity of degradation. Specifically, Perceive-IR contains two stages: (1) prompt learning stage and (2) restoration stage. In the prompt learning stage, we leverage prompt learning to acquire a fine-grained quality perceiver capable of distinguishing three-tier quality levels by constraining the prompt-image similarity in the CLIP perception space. Subsequently, this quality perceiver and difficulty-adaptive perceptual loss are integrated as a quality-aware learning strategy to realize fine-grained quality control in restoration stage. For the restoration stage, a semantic guidance module (SGM) and compact feature extraction (CFE) are proposed to further promote the restoration process by utilizing the robust semantic information from the pre-trained large scale vision models and distinguishing degradation-specific features. Extensive experiments demonstrate that our Perceive-IR outperforms state-of-the-art methods in all-in-one image restoration tasks and exhibit superior generalization ability when dealing with unseen tasks.

Related papers

• PromptHSI: Universal Hyperspectral Image Restoration Framework for Composite Degradation [28.10512516485237]
  We propose PromptHSI, the first universal AiO HSI restoration framework. We decompose text prompts into intensity and bias controllers to guide the restoration process. Our unified architecture excels at both fine-grained recovery and global information restoration tasks.
  arXiv  Detail & Related papers  (2024-11-24T17:08:58Z)
• Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model [55.46927355649013]
  We introduce a novel Multi-modal Guided Real-World Face Restoration technique. MGFR can mitigate the generation of false facial attributes and identities. We present the Reface-HQ dataset, comprising over 23,000 high-resolution facial images across 5,000 identities.
  arXiv  Detail & Related papers  (2024-10-05T13:46:56Z)
• Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration [19.87693298262894]
  We propose Diff-Restorer, a universal image restoration method based on the diffusion model. We utilize the pre-trained visual language model to extract visual prompts from degraded images. We also design a Degradation-aware Decoder to perform structural correction and convert the latent code to the pixel domain.
  arXiv  Detail & Related papers  (2024-07-04T05:01:10Z)
• SPIRE: Semantic Prompt-Driven Image Restoration [66.26165625929747]
  We develop SPIRE, a Semantic and restoration Prompt-driven Image Restoration framework. Our approach is the first framework that supports fine-level instruction through language-based quantitative specification of the restoration strength. Our experiments demonstrate the superior restoration performance of SPIRE compared to the state of the arts.
  arXiv  Detail & Related papers  (2023-12-18T17:02:30Z)
• Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration [58.11518043688793]
  MPerceiver is a novel approach to enhance adaptiveness, generalizability and fidelity for all-in-one image restoration. MPerceiver is trained on 9 tasks for all-in-one IR and outperforms state-of-the-art task-specific methods across most tasks.
  arXiv  Detail & Related papers  (2023-12-05T17:47:11Z)
• All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation [47.00239809958627]
  We propose a novel All-in-one Multi-degradation Image Restoration Network (AMIRNet) AMIRNet learns a degradation representation for unknown degraded images by progressively constructing a tree structure through clustering. This tree-structured representation explicitly reflects the consistency and discrepancy of various distortions, providing a specific clue for image restoration.
  arXiv  Detail & Related papers  (2023-08-06T04:51:41Z)
• PromptIR: Prompting for All-in-One Blind Image Restoration [64.02374293256001]
  We present a prompt-based learning approach, PromptIR, for All-In-One image restoration. Our method uses prompts to encode degradation-specific information, which is then used to dynamically guide the restoration network. PromptIR offers a generic and efficient plugin module with few lightweight prompts.
  arXiv  Detail & Related papers  (2023-06-22T17:59:52Z)
• Implicit Subspace Prior Learning for Dual-Blind Face Restoration [66.67059961379923]
  A novel implicit subspace prior learning (ISPL) framework is proposed as a generic solution to dual-blind face restoration. Experimental results demonstrate significant perception-distortion improvement of ISPL against existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-12T08:04:24Z)
• Blind Image Restoration without Prior Knowledge [0.22940141855172028]
  We present the Self-Normalization Side-Chain (SCNC), a novel approach to blind universal restoration in which no prior knowledge of the degradation is needed. The SCNC can be added to any existing CNN topology, and is trained along with the rest of the network in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-03-03T19:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.