Image Restoration using Feature-guidance

• URL: http://arxiv.org/abs/2201.00187v1
• Date: Sat, 1 Jan 2022 13:10:19 GMT
• Title: Image Restoration using Feature-guidance
• Authors: Maitreya Suin, Kuldeep Purohit, A. N. Rajagopalan
• Abstract summary: We present a new approach suitable for handling the image-specific and spatially-varying nature of degradation in images. We decompose the restoration task into two stages of degradation localization and degraded region-guided restoration. We demonstrate that the model trained for this auxiliary task contains vital region knowledge, which can be exploited to guide the restoration network's training.
• Score: 43.02281823557039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image restoration is the task of recovering a clean image from a degraded version. In most cases, the degradation is spatially varying, and it requires the restoration network to both localize and restore the affected regions. In this paper, we present a new approach suitable for handling the image-specific and spatially-varying nature of degradation in images affected by practically occurring artifacts such as blur, rain-streaks. We decompose the restoration task into two stages of degradation localization and degraded region-guided restoration, unlike existing methods which directly learn a mapping between the degraded and clean images. Our premise is to use the auxiliary task of degradation mask prediction to guide the restoration process. We demonstrate that the model trained for this auxiliary task contains vital region knowledge, which can be exploited to guide the restoration network's training using attentive knowledge distillation technique. Further, we propose mask-guided convolution and global context aggregation module that focuses solely on restoring the degraded regions. The proposed approach's effectiveness is demonstrated by achieving significant improvement over strong baselines.

Related papers

• Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration [19.87693298262894]
  We propose Diff-Restorer, a universal image restoration method based on the diffusion model. We utilize the pre-trained visual language model to extract visual prompts from degraded images. We also design a Degradation-aware Decoder to perform structural correction and convert the latent code to the pixel domain.
  arXiv  Detail & Related papers  (2024-07-04T05:01:10Z)
• Repositioning the Subject within Image [78.8467524191102]
  We introduce an innovative dynamic manipulation task, subject repositioning. This task involves relocating a user-specified subject to a desired position while preserving the image's fidelity. Our research reveals that the fundamental sub-tasks of subject repositioning can be effectively reformulated as a unified, prompt-guided inpainting task.
  arXiv  Detail & Related papers  (2024-01-30T10:04:49Z)
• Improving Image Restoration through Removing Degradations in Textual Representations [60.79045963573341]
  We introduce a new perspective for improving image restoration by removing degradation in the textual representations of a degraded image. To address the cross-modal assistance, we propose to map the degraded images into textual representations for removing the degradations. In particular, We ingeniously embed an image-to-text mapper and text restoration module into CLIP-equipped text-to-image models to generate the guidance.
  arXiv  Detail & Related papers  (2023-12-28T19:18:17Z)
• SPIRE: Semantic Prompt-Driven Image Restoration [66.26165625929747]
  We develop SPIRE, a Semantic and restoration Prompt-driven Image Restoration framework. Our approach is the first framework that supports fine-level instruction through language-based quantitative specification of the restoration strength. Our experiments demonstrate the superior restoration performance of SPIRE compared to the state of the arts.
  arXiv  Detail & Related papers  (2023-12-18T17:02:30Z)
• All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation [47.00239809958627]
  We propose a novel All-in-one Multi-degradation Image Restoration Network (AMIRNet) AMIRNet learns a degradation representation for unknown degraded images by progressively constructing a tree structure through clustering. This tree-structured representation explicitly reflects the consistency and discrepancy of various distortions, providing a specific clue for image restoration.
  arXiv  Detail & Related papers  (2023-08-06T04:51:41Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Blind Image Restoration without Prior Knowledge [0.22940141855172028]
  We present the Self-Normalization Side-Chain (SCNC), a novel approach to blind universal restoration in which no prior knowledge of the degradation is needed. The SCNC can be added to any existing CNN topology, and is trained along with the rest of the network in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-03-03T19:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.