Learning Enriched Features for Fast Image Restoration and Enhancement

• URL: http://arxiv.org/abs/2205.01649v1
• Date: Tue, 19 Apr 2022 17:59:45 GMT
• Title: Learning Enriched Features for Fast Image Restoration and Enhancement
• Authors: Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, Ling Shao
• Abstract summary: This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
• Score: 166.17296369600774
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks (CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatial details are preserved but the contextual information cannot be precisely encoded. In the latter case, generated outputs are semantically reliable but spatially less accurate. This paper presents a new architecture with a holistic goal of maintaining spatially-precise high-resolution representations through the entire network, and receiving complementary contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing the following key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) non-local attention mechanism for capturing contextual information, and (d) attention based multi-scale feature aggregation. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on six real image benchmark datasets demonstrate that our method, named as MIRNet-v2 , achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at https://github.com/swz30/MIRNetv2

Related papers

• Dilated Strip Attention Network for Image Restoration [5.65781374269726]
  We propose a dilated strip attention network (DSAN) for image restoration. By employing the DSA operation horizontally and vertically, each location can harvest the contextual information from a much wider region. Our experiments show that our DSAN outperforms state-of-the-art algorithms on several image restoration tasks.
  arXiv  Detail & Related papers  (2024-07-26T09:12:30Z)
• MRIR: Integrating Multimodal Insights for Diffusion-based Realistic Image Restoration [17.47612023350466]
  We propose MRIR, a diffusion-based restoration method with multimodal insights. For the textual level, we harness the power of the pre-trained multimodal large language model to infer meaningful semantic information from low-quality images. For the visual level, we mainly focus on the pixel level control. Thus, we utilize a Pixel-level Processor and ControlNet to control spatial structures.
  arXiv  Detail & Related papers  (2024-07-04T04:55:14Z)
• Super-Resolving Face Image by Facial Parsing Information [52.1267613768555]
  Face super-resolution is a technology that transforms a low-resolution face image into the corresponding high-resolution one. We build a novel parsing map guided face super-resolution network which extracts the face prior from low-resolution face image. High-resolution features contain more precise spatial information while low-resolution features provide strong contextual information.
  arXiv  Detail & Related papers  (2023-04-06T08:19:03Z)
• Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
  convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data. Transformers have shown significant performance gains on natural language and high-level vision tasks. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:10Z)
• Multi-Stage Progressive Image Restoration [167.6852235432918]
  We propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets.
  arXiv  Detail & Related papers  (2021-02-04T18:57:07Z)
• Learning Deep Interleaved Networks with Asymmetric Co-Attention for Image Restoration [65.11022516031463]
  We present a deep interleaved network (DIN) that learns how information at different states should be combined for high-quality (HQ) images reconstruction. In this paper, we propose asymmetric co-attention (AsyCA) which is attached at each interleaved node to model the feature dependencies. Our presented DIN can be trained end-to-end and applied to various image restoration tasks.
  arXiv  Detail & Related papers  (2020-10-29T15:32:00Z)
• Multi-image Super Resolution of Remotely Sensed Images using Residual Feature Attention Deep Neural Networks [1.3764085113103222]
  The presented research proposes a novel residual attention model (RAMS) that efficiently tackles the multi-image super-resolution task. We introduce the mechanism of visual feature attention with 3D convolutions in order to obtain an aware data fusion and information extraction. Our representation learning network makes extensive use of nestled residual connections to let flow redundant low-frequency signals.
  arXiv  Detail & Related papers  (2020-07-06T22:54:02Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.