Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution

• URL: http://arxiv.org/abs/2108.05302v1
• Date: Wed, 11 Aug 2021 16:11:17 GMT
• Title: Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution
• Authors: Jingyun Liang, Guolei Sun, Kai Zhang, Luc Van Gool, Radu Timofte
• Abstract summary: Existing blind image super-resolution (SR) methods mostly assume blur kernels are spatially invariant across the whole image. This paper proposes a mutual affine network (MANet) for spatially variant kernel estimation.
• Score: 130.32026819172256
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Existing blind image super-resolution (SR) methods mostly assume blur kernels are spatially invariant across the whole image. However, such an assumption is rarely applicable for real images whose blur kernels are usually spatially variant due to factors such as object motion and out-of-focus. Hence, existing blind SR methods would inevitably give rise to poor performance in real applications. To address this issue, this paper proposes a mutual affine network (MANet) for spatially variant kernel estimation. Specifically, MANet has two distinctive features. First, it has a moderate receptive field so as to keep the locality of degradation. Second, it involves a new mutual affine convolution (MAConv) layer that enhances feature expressiveness without increasing receptive field, model size and computation burden. This is made possible through exploiting channel interdependence, which applies each channel split with an affine transformation module whose input are the rest channel splits. Extensive experiments on synthetic and real images show that the proposed MANet not only performs favorably for both spatially variant and invariant kernel estimation, but also leads to state-of-the-art blind SR performance when combined with non-blind SR methods.

Related papers

• BlindDiff: Empowering Degradation Modelling in Diffusion Models for Blind Image Super-Resolution [52.47005445345593]
  BlindDiff is a DM-based blind SR method to tackle the blind degradation settings in SISR. BlindDiff seamlessly integrates the MAP-based optimization into DMs. Experiments on both synthetic and real-world datasets show that BlindDiff achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-03-15T11:21:34Z)
• Adaptive Multi-modal Fusion of Spatially Variant Kernel Refinement with Diffusion Model for Blind Image Super-Resolution [23.34647832483913]
  We introduce a framework known as Adaptive Multi-modal Fusion of textbfSpatially Variant Kernel Refinement with Diffusion Model for Blind Image textbfSuper-textbfResolution. We also introduce the Adaptive Multi-Modal Fusion (AMF) module to align the information from three modalities: low-resolution images, depth maps, and blur kernels.
  arXiv  Detail & Related papers  (2024-03-09T06:01:25Z)
• Better "CMOS" Produces Clearer Images: Learning Space-Variant Blur Estimation for Blind Image Super-Resolution [30.816546273417774]
  We introduce two new datasets with out-of-focus blur, i.e., NYUv2-BSR and Cityscapes-BSR, to support further researches of blind SR with space-variant blur. Based on the datasets, we design a novel Cross-MOdal fuSion network (CMOS) that estimate both blur and semantics simultaneously.
  arXiv  Detail & Related papers  (2023-04-07T08:40:31Z)
• Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment [53.401889855278704]
  Few-shot fine-grained recognition (FS-FGR) aims to recognize novel fine-grained categories with the help of limited available samples. We propose a two-stage background suppression and foreground alignment framework, which is composed of a background activation suppression (BAS) module, a foreground object alignment (FOA) module, and a local to local (L2L) similarity metric. Experiments conducted on multiple popular fine-grained benchmarks demonstrate that our method outperforms the existing state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2022-10-04T07:54:40Z)
• Spectrum-to-Kernel Translation for Accurate Blind Image Super-Resolution [33.59749785182318]
  We propose a novel blind SR framework to super-resolve LR images degraded by arbitrary blur kernel. We first demonstrate that feature representation in frequency domain is more conducive for blur kernel reconstruction than in spatial domain. Experiments on both synthetic and real-world images demonstrate that our proposed method sufficiently reduces blur kernel estimation error.
  arXiv  Detail & Related papers  (2021-10-23T06:03:22Z)
• Blind Image Super-Resolution via Contrastive Representation Learning [41.17072720686262]
  We design a contrastive representation learning network that focuses on blind SR of images with multi-modal and spatially variant distributions. We show that the proposed CRL-SR can handle multi-modal and spatially variant degradation effectively under blind settings. It also outperforms state-of-the-art SR methods qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2021-07-01T19:34:23Z)
• SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation [65.88321755969677]
  Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution. Recent approaches extend the scope to real-valued upsampling factors. We propose the SRWarp framework to further generalize the SR tasks toward an arbitrary image transformation.
  arXiv  Detail & Related papers  (2021-04-21T02:50:41Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.