Integrating Spatial and Frequency Information for Under-Display Camera Image Restoration

• URL: http://arxiv.org/abs/2501.18517v1
• Date: Thu, 30 Jan 2025 17:30:00 GMT
• Title: Integrating Spatial and Frequency Information for Under-Display Camera Image Restoration
• Authors: Kyusu Ahn, Jinpyo Kim, Chanwoo Park, JiSoo Kim, Jaejin Lee,
• Abstract summary: Under-Display Camera (UDC) houses a digital camera lens under a display panel.<n>UDC introduces complex degradations such as noise, blur, decrease in transmittance, and flare.<n>We propose a novel multi-level architecture called SFIM to restore UDC-distorted images.
• Score: 5.696863540133448
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Under-Display Camera (UDC) houses a digital camera lens under a display panel. However, UDC introduces complex degradations such as noise, blur, decrease in transmittance, and flare. Despite the remarkable progress, previous research on UDC mainly focuses on eliminating diffraction in the spatial domain and rarely explores its potential in the frequency domain. It is essential to consider both the spatial and frequency domains effectively. For example, degradations, such as noise and blur, can be addressed by local information (e.g., CNN kernels in the spatial domain). At the same time, tackling flares may require leveraging global information (e.g., the frequency domain). In this paper, we revisit the UDC degradations in the Fourier space and figure out intrinsic frequency priors that imply the presence of the flares. Based on this observation, we propose a novel multi-level DNN architecture called SFIM. It efficiently restores UDC-distorted images by integrating local and global (the collective contribution of all points in the image) information. The architecture exploits CNNs to capture local information and FFT-based models to capture global information. SFIM comprises a spatial domain block (SDB), a Frequency Domain Block (FDB), and an Attention-based Multi-level Integration Block (AMIB). Specifically, SDB focuses more on detailed textures such as noise and blur, FDB emphasizes irregular texture loss in extensive areas such as flare, and AMIB enables effective cross-domain interaction. SFIM's superior performance over state-of-the-art approaches is demonstrated through rigorous quantitative and qualitative assessments across three UDC benchmarks.

Related papers

• Wavelet-Guided Dual-Frequency Encoding for Remote Sensing Change Detection [67.84730634802204]
  Change detection in remote sensing imagery plays a vital role in various engineering applications, such as natural disaster monitoring, urban expansion tracking, and infrastructure management.<n>Most existing methods still rely on spatial-domain modeling, where the limited diversity of feature representations hinders the detection of subtle change regions.<n>We observe that frequency-domain feature modeling particularly in the wavelet domain amplify fine-grained differences in frequency components, enhancing the perception of edge changes that are challenging to capture in the spatial domain.
  arXiv  Detail & Related papers  (2025-08-07T11:14:16Z)
• DFDNet: Dynamic Frequency-Guided De-Flare Network [8.713784455593778]
  This paper presents a novel dynamic frequency-guided deflare network (DFDNet) that decouples content information from flare artifacts in the frequency domain.<n>The experimental results demonstrate that the proposed method outperforms existing state-of-the-art methods in terms of performance.
  arXiv  Detail & Related papers  (2025-07-23T13:14:59Z)
• Learning Multi-scale Spatial-frequency Features for Image Denoising [58.883244886588336]
  We propose a novel multi-scale adaptive dual-domain network (MADNet) for image denoising.<n>We use image pyramid inputs to restore noise-free results from low-resolution images.<n>In order to realize the interaction of high-frequency and low-frequency information, we design an adaptive spatial-frequency learning unit.
  arXiv  Detail & Related papers  (2025-06-19T13:28:09Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• FUSION: Frequency-guided Underwater Spatial Image recOnstructioN [0.0]
  Underwater images suffer from severe degradations, including color distortions, reduced visibility, and loss of structural details due to wavelength-dependent attenuation and scattering. Existing enhancement methods primarily focus on spatial-domain processing, neglecting the frequency domain's potential to capture global color distributions and long-range dependencies. We propose fusion, a dual-domain deep learning framework that jointly leverages spatial and frequency domain information.
  arXiv  Detail & Related papers  (2025-04-01T23:16:19Z)
• Spatial and Frequency Domain Adaptive Fusion Network for Image Deblurring [0.0]
  Image deblurring aims to reconstruct a latent sharp image from its corresponding blurred one. We propose a spatial-frequency domain adaptive fusion network (SFAFNet) to address this limitation. Our SFAFNet performs favorably compared to state-of-the-art approaches on commonly used benchmarks.
  arXiv  Detail & Related papers  (2025-02-20T02:43:55Z)
• United Domain Cognition Network for Salient Object Detection in Optical Remote Sensing Images [21.76732661032257]
  We propose a novel United Domain Cognition Network (UDCNet) to jointly explore the global-local information in the frequency and spatial domains. Experimental results demonstrate the superiority of the proposed UDCNet over 24 state-of-the-art models.
  arXiv  Detail & Related papers  (2024-11-11T04:12:27Z)
• Channel-Partitioned Windowed Attention And Frequency Learning for Single Image Super-Resolution [1.8506868409351092]
  Window-based attention methods have shown great potential for computer vision tasks, particularly in Single Image Super-Resolution (SISR) We propose a new Channel-Partitioned Attention Transformer (CPAT) to better capture long-range dependencies by sequentially expanding windows along the height and width of feature maps. In addition, we propose a novel Spatial-Frequency Interaction Module (SFIM), which incorporates information from spatial and frequency domains to provide a more comprehensive information from feature maps.
  arXiv  Detail & Related papers  (2024-07-23T07:17:10Z)
• Emphasizing Crucial Features for Efficient Image Restoration [6.204240924744974]
  We propose a framework to adapt to varying degrees of degradation across different regions for image restoration. Specifically, we design a spatial and frequency attention mechanism (SFAM) to emphasize crucial features for restoration. We also propose our ECFNet, which integrates the aforementioned components into a U-shaped backbone for recovering high-quality images.
  arXiv  Detail & Related papers  (2024-05-19T07:04:05Z)
• FDCE-Net: Underwater Image Enhancement with Embedding Frequency and Dual Color Encoder [49.79611204954311]
  Underwater images often suffer from various issues such as low brightness, color shift, blurred details, and noise due to absorption light and scattering caused by water and suspended particles. Previous underwater image enhancement (UIE) methods have primarily focused on spatial domain enhancement, neglecting the frequency domain information inherent in the images.
  arXiv  Detail & Related papers  (2024-04-27T15:16:34Z)
• Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising [94.09442506816724]
  Blind-spot networks (BSN) have been prevalent neural architectures in self-supervised image denoising (SSID)<n>We build a Transformer-based Blind-Spot Network (TBSN) which shows strong local fitting and global perspective abilities.
  arXiv  Detail & Related papers  (2024-04-11T15:39:10Z)
• Frequency Perception Network for Camouflaged Object Detection [51.26386921922031]
  We propose a novel learnable and separable frequency perception mechanism driven by the semantic hierarchy in the frequency domain. Our entire network adopts a two-stage model, including a frequency-guided coarse localization stage and a detail-preserving fine localization stage. Compared with the currently existing models, our proposed method achieves competitive performance in three popular benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-17T11:30:46Z)
• Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Models [89.76587063609806]
  We study the denoising diffusion probabilistic model (DDPM) in wavelet space, instead of pixel space, for visual synthesis. By explicitly modeling the wavelet signals, we find our model is able to generate images with higher quality on several datasets.
  arXiv  Detail & Related papers  (2023-07-27T06:53:16Z)
• Wavelet-Based Network For High Dynamic Range Imaging [64.66969585951207]
  Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal. In this work, we propose a novel frequency-guided end-to-end deep neural network (FNet) to conduct HDR fusion in the frequency domain, and Wavelet Transform (DWT) is used to decompose inputs into different frequency bands. The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details.
  arXiv  Detail & Related papers  (2021-08-03T12:26:33Z)
• Low Light Image Enhancement via Global and Local Context Modeling [164.85287246243956]
  We introduce a context-aware deep network for low-light image enhancement. First, it features a global context module that models spatial correlations to find complementary cues over full spatial domain. Second, it introduces a dense residual block that captures local context with a relatively large receptive field.
  arXiv  Detail & Related papers  (2021-01-04T09:40:54Z)
• Frequency Consistent Adaptation for Real World Super Resolution [64.91914552787668]
  We propose a novel Frequency Consistent Adaptation (FCA) that ensures the frequency domain consistency when applying Super-Resolution (SR) methods to the real scene. We estimate degradation kernels from unsupervised images and generate the corresponding Low-Resolution (LR) images. Based on the domain-consistent LR-HR pairs, we train easy-implemented Convolutional Neural Network (CNN) SR models.
  arXiv  Detail & Related papers  (2020-12-18T08:25:39Z)
• Frequency Domain Image Translation: More Photo-realistic, Better Identity-preserving [36.606114597585396]
  We propose a novel frequency domain image translation framework, exploiting frequency information for enhancing the image generation process. Our key idea is to decompose the image into low-frequency and high-frequency components, where the high-frequency feature captures object structure akin to the identity. Extensive experiments and ablations show that FDIT effectively preserves the identity of the source image, and produces photo-realistic images.
  arXiv  Detail & Related papers  (2020-11-27T08:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.