Dual-domain Modulation Network for Lightweight Image Super-Resolution

• URL: http://arxiv.org/abs/2503.10047v1
• Date: Thu, 13 Mar 2025 04:59:46 GMT
• Title: Dual-domain Modulation Network for Lightweight Image Super-Resolution
• Authors: Wenjie Li, Heng Guo, Yuefeng Hou, Guangwei Gao, Zhanyu Ma,
• Abstract summary: Lightweight image super-resolution (SR) aims to reconstruct high-resolution images from low-resolution images with limited computational costs.<n>Existing frequency-based SR methods cannot balance the reconstruction of overall structures and high-frequency parts.<n>We show introducing both wavelet and Fourier information allowing our model to consider both high-frequency features and overall SR structure reconstruction.
• Score: 26.992373105057684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lightweight image super-resolution (SR) aims to reconstruct high-resolution images from low-resolution images with limited computational costs. We find existing frequency-based SR methods cannot balance the reconstruction of overall structures and high-frequency parts. Meanwhile, these methods are inefficient for handling frequency features and unsuitable for lightweight SR. In this paper, we show introducing both wavelet and Fourier information allows our model to consider both high-frequency features and overall SR structure reconstruction while reducing costs. Specifically, we propose a dual-domain modulation network that utilize wavelet-domain modulation self-Transformer (WMT) plus Fourier supervision to modulate frequency features in addition to spatial domain modulation. Compared to existing frequency-based SR modules, our WMT is more suitable for frequency learning in lightweight SR. Experimental results show that our method achieves a comparable PSNR of SRFormer and MambaIR while with less than 50% and 60% of their FLOPs and achieving inference speeds 15.4x and 5.4x faster, respectively, demonstrating the effectiveness of our method on SR quality and lightweight. Codes will be released upon acceptance.

Related papers

• FreqINR: Frequency Consistency for Implicit Neural Representation with Adaptive DCT Frequency Loss [5.349799154834945]
  This paper introduces Frequency Consistency for Implicit Neural Representation (FreqINR), an innovative Arbitrary-scale Super-resolution method. During training, we employ Adaptive Discrete Cosine Transform Frequency Loss (ADFL) to minimize the frequency gap between HR and ground-truth images. During inference, we extend the receptive field to preserve spectral coherence between low-resolution (LR) and ground-truth images.
  arXiv  Detail & Related papers  (2024-08-25T03:53:17Z)
• Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution [49.902047563260496]
  We develop the first attempt to integrate the Vision State Space Model (Mamba) for remote sensing image (RSI) super-resolution. To achieve better SR reconstruction, building upon Mamba, we devise a Frequency-assisted Mamba framework, dubbed FMSR. Our FMSR features a multi-level fusion architecture equipped with the Frequency Selection Module (FSM), Vision State Space Module (VSSM), and Hybrid Gate Module (HGM)
  arXiv  Detail & Related papers  (2024-05-08T11:09:24Z)
• A Scale-Arbitrary Image Super-Resolution Network Using Frequency-domain Information [42.55177009667711]
  Image super-resolution (SR) is a technique to recover lost high-frequency information in low-resolution (LR) images. In this paper, we study image features in the frequency domain to design a novel scale-arbitrary image SR network.
  arXiv  Detail & Related papers  (2022-12-08T15:10:49Z)
• Cross-Modality High-Frequency Transformer for MR Image Super-Resolution [100.50972513285598]
  We build an early effort to build a Transformer-based MR image super-resolution framework. We consider two-fold domain priors including the high-frequency structure prior and the inter-modality context prior. We establish a novel Transformer architecture, called Cross-modality high-frequency Transformer (Cohf-T), to introduce such priors into super-resolving the low-resolution images.
  arXiv  Detail & Related papers  (2022-03-29T07:56:55Z)
• Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization [67.99082021804145]
  We propose an explicit solution to the COO problem, called Detail Enhanced Contrastive Loss (DECLoss) DECLoss utilizes the clustering property of contrastive learning to directly reduce the variance of the potential high-resolution distribution. We evaluate DECLoss on multiple super-resolution benchmarks and demonstrate that it improves the perceptual quality of PSNR-oriented models.
  arXiv  Detail & Related papers  (2022-01-04T08:30:09Z)
• FreqNet: A Frequency-domain Image Super-Resolution Network with Dicrete Cosine Transform [16.439669339293747]
  Single image super-resolution(SISR) is an ill-posed problem that aims to obtain high-resolution (HR) output from low-resolution (LR) input. Despite the high peak signal-to-noise ratios(PSNR) results, it is difficult to determine whether the model correctly adds desired high-frequency details. We propose FreqNet, an intuitive pipeline from the frequency domain perspective, to solve this problem.
  arXiv  Detail & Related papers  (2021-11-21T11:49:12Z)
• MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution [74.24676600271253]
  We propose the MASA network for RefSR, where two novel modules are designed to address these problems. The proposed Match & Extraction Module significantly reduces the computational cost by a coarse-to-fine correspondence matching scheme. The Spatial Adaptation Module learns the difference of distribution between the LR and Ref images, and remaps the distribution of Ref features to that of LR features in a spatially adaptive way.
  arXiv  Detail & Related papers  (2021-06-04T07:15:32Z)
• Fourier Space Losses for Efficient Perceptual Image Super-Resolution [131.50099891772598]
  We show that it is possible to improve the performance of a recently introduced efficient generator architecture solely with the application of our proposed loss functions. We show that our losses' direct emphasis on the frequencies in Fourier-space significantly boosts the perceptual image quality. The trained generator achieves comparable results with and is 2.4x and 48x faster than state-of-the-art perceptual SR methods RankSRGAN and SRFlow respectively.
  arXiv  Detail & Related papers  (2021-06-01T20:34:52Z)
• Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
  This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
  arXiv  Detail & Related papers  (2021-03-15T12:54:26Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.