WaveFill: A Wavelet-based Generation Network for Image Inpainting

• URL: http://arxiv.org/abs/2107.11027v1
• Date: Fri, 23 Jul 2021 04:44:40 GMT
• Title: WaveFill: A Wavelet-based Generation Network for Image Inpainting
• Authors: Yingchen Yu, Fangneng Zhan, Shijian Lu, Jianxiong Pan, Feiying Ma, Xuansong Xie, Chunyan Miao
• Abstract summary: WaveFill is a wavelet-based inpainting network that decomposes images into multiple frequency bands. WaveFill decomposes images by using discrete wavelet transform (DWT) that preserves spatial information naturally. It applies L1 reconstruction loss to the low-frequency bands and adversarial loss to high-frequency bands, hence effectively mitigate inter-frequency conflicts.
• Score: 57.012173791320855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image inpainting aims to complete the missing or corrupted regions of images with realistic contents. The prevalent approaches adopt a hybrid objective of reconstruction and perceptual quality by using generative adversarial networks. However, the reconstruction loss and adversarial loss focus on synthesizing contents of different frequencies and simply applying them together often leads to inter-frequency conflicts and compromised inpainting. This paper presents WaveFill, a wavelet-based inpainting network that decomposes images into multiple frequency bands and fills the missing regions in each frequency band separately and explicitly. WaveFill decomposes images by using discrete wavelet transform (DWT) that preserves spatial information naturally. It applies L1 reconstruction loss to the decomposed low-frequency bands and adversarial loss to high-frequency bands, hence effectively mitigate inter-frequency conflicts while completing images in spatial domain. To address the inpainting inconsistency in different frequency bands and fuse features with distinct statistics, we design a novel normalization scheme that aligns and fuses the multi-frequency features effectively. Extensive experiments over multiple datasets show that WaveFill achieves superior image inpainting qualitatively and quantitatively.

Related papers

• Zero-Shot Low-Light Image Enhancement via Joint Frequency Domain Priors Guided Diffusion [2.3874115898130865]
  We will propose a new zero-shot low-light enhancement method to compensate for the lack of light and structural information in the diffusion sampling process. The inspiration comes from the similarity between the wavelet and Fourier frequency domains. Sufficient experiments show that the framework is robust and effective in various scenarios.
  arXiv  Detail & Related papers  (2024-11-21T09:16:51Z)
• Adaptive Frequency Enhancement Network for Single Image Deraining [10.64622976628013]
  We introduce a novel end-to-end Adaptive Frequency Enhancement Network (AFENet) specifically for single image deraining. We employ convolutions of different scales to adaptively decompose image frequency bands, introduce a feature enhancement module, and present a novel interaction module. This approach empowers the deraining network to eliminate diverse and complex rainy patterns and to reconstruct image details accurately.
  arXiv  Detail & Related papers  (2024-07-19T13:24:05Z)
• FDCE-Net: Underwater Image Enhancement with Embedding Frequency and Dual Color Encoder [49.79611204954311]
  Underwater images often suffer from various issues such as low brightness, color shift, blurred details, and noise due to absorption light and scattering caused by water and suspended particles. Previous underwater image enhancement (UIE) methods have primarily focused on spatial domain enhancement, neglecting the frequency domain information inherent in the images.
  arXiv  Detail & Related papers  (2024-04-27T15:16:34Z)
• Both Spatial and Frequency Cues Contribute to High-Fidelity Image Inpainting [9.080472817672263]
  Deep generative approaches have obtained great success in image inpainting recently. Most generative inpainting networks suffer from either over-smooth results or aliasing artifacts. We propose an effective Frequency-Spatial Complementary Network (FSCN) by exploiting rich semantic information in both spatial and frequency domains.
  arXiv  Detail & Related papers  (2023-07-15T01:52:06Z)
• Exploring Inter-frequency Guidance of Image for Lightweight Gaussian Denoising [1.52292571922932]
  We propose a novel network architecture denoted as IGNet, in order to refine the frequency bands from low to high in a progressive manner. With this design, more inter-frequency prior and information are utilized, thus the model size can be lightened while still perserves competitive results.
  arXiv  Detail & Related papers  (2021-12-22T10:35:53Z)
• Wavelet-Based Network For High Dynamic Range Imaging [64.66969585951207]
  Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal. In this work, we propose a novel frequency-guided end-to-end deep neural network (FNet) to conduct HDR fusion in the frequency domain, and Wavelet Transform (DWT) is used to decompose inputs into different frequency bands. The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details.
  arXiv  Detail & Related papers  (2021-08-03T12:26:33Z)
• WaveTransform: Crafting Adversarial Examples via Input Decomposition [69.01794414018603]
  We introduce WaveTransform', that creates adversarial noise corresponding to low-frequency and high-frequency subbands, separately (or in combination) Experiments show that the proposed attack is effective against the defense algorithm and is also transferable across CNNs.
  arXiv  Detail & Related papers  (2020-10-29T17:16:59Z)
• Wavelet Channel Attention Module with a Fusion Network for Single Image Deraining [46.62290347397139]
  Single image deraining is a crucial problem because rain severely degenerates the visibility of images. We propose the new convolutional neural network (CNN) called the wavelet channel attention module with a fusion network.
  arXiv  Detail & Related papers  (2020-07-17T18:06:13Z)
• Wavelet Integrated CNNs for Noise-Robust Image Classification [51.18193090255933]
  We enhance CNNs by replacing max-pooling, strided-convolution, and average-pooling with Discrete Wavelet Transform (DWT) WaveCNets, the wavelet integrated versions of VGG, ResNets, and DenseNet, achieve higher accuracy and better noise-robustness than their vanilla versions.
  arXiv  Detail & Related papers  (2020-05-07T09:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.