Wavelet Prior Attention Learning in Axial Inpainting Network
- URL: http://arxiv.org/abs/2206.03113v1
- Date: Tue, 7 Jun 2022 08:45:27 GMT
- Title: Wavelet Prior Attention Learning in Axial Inpainting Network
- Authors: Chenjie Cao, Chengrong Wang, Yuntao Zhang, Yanwei Fu
- Abstract summary: We propose a novel model -- Wavelet prior attention learning in Axial Inpainting Network (WAIN)
The WPA guides the high-level feature aggregation in the multi-scale frequency domain, alleviating the textual artifacts.
Stacked ATs employ unmasked clues to help model reasonable features along with low-level features of horizontal and vertical axes.
- Score: 35.06912946192495
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Image inpainting is the task of filling masked or unknown regions of an image
with visually realistic contents, which has been remarkably improved by Deep
Neural Networks (DNNs) recently. Essentially, as an inverse problem, the
inpainting has the underlying challenges of reconstructing semantically
coherent results without texture artifacts. Many previous efforts have been
made via exploiting attention mechanisms and prior knowledge, such as edges and
semantic segmentation. However, these works are still limited in practice by an
avalanche of learnable prior parameters and prohibitive computational burden.
To this end, we propose a novel model -- Wavelet prior attention learning in
Axial Inpainting Network (WAIN), whose generator contains the encoder, decoder,
as well as two key components of Wavelet image Prior Attention (WPA) and
stacked multi-layer Axial-Transformers (ATs). Particularly, the WPA guides the
high-level feature aggregation in the multi-scale frequency domain, alleviating
the textual artifacts. Stacked ATs employ unmasked clues to help model
reasonable features along with low-level features of horizontal and vertical
axes, improving the semantic coherence. Extensive quantitative and qualitative
experiments on Celeba-HQ and Places2 datasets are conducted to validate that
our WAIN can achieve state-of-the-art performance over the competitors. The
codes and models will be released.
Related papers
- Distance Weighted Trans Network for Image Completion [52.318730994423106]
We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components.
CNNs are used to augment the local texture information of coarse priors.
DWT blocks are used to recover certain coarse textures and coherent visual structures.
arXiv Detail & Related papers (2023-10-11T12:46:11Z) - Both Spatial and Frequency Cues Contribute to High-Fidelity Image
Inpainting [9.080472817672263]
Deep generative approaches have obtained great success in image inpainting recently.
Most generative inpainting networks suffer from either over-smooth results or aliasing artifacts.
We propose an effective Frequency-Spatial Complementary Network (FSCN) by exploiting rich semantic information in both spatial and frequency domains.
arXiv Detail & Related papers (2023-07-15T01:52:06Z) - Semantic-aware Texture-Structure Feature Collaboration for Underwater
Image Enhancement [58.075720488942125]
Underwater image enhancement has become an attractive topic as a significant technology in marine engineering and aquatic robotics.
We develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model.
We also apply the proposed algorithm to the underwater salient object detection task to reveal the favorable semantic-aware ability for high-level vision tasks.
arXiv Detail & Related papers (2022-11-19T07:50:34Z) - Progressive with Purpose: Guiding Progressive Inpainting DNNs through
Context and Structure [0.0]
We propose a novel inpainting network that maintains the structural and contextual integrity of a processed image.
Inspired by the Gaussian and Laplacian pyramids, the core of the proposed network is a feature extraction module named GLE.
Our benchmarking experiments demonstrate that the proposed method achieves clear improvement in performance over many state-of-the-art inpainting algorithms.
arXiv Detail & Related papers (2022-09-21T02:15:02Z) - High-Fidelity Image Inpainting with GAN Inversion [23.49170140410603]
In this paper, we propose a novel GAN inversion model for image inpainting, dubbed InvertFill.
Within the encoder, the pre-modulation network leverages multi-scale structures to encode more discriminative semantics into style vectors.
To reconstruct faithful and photorealistic images, a simple yet effective Soft-update Mean Latent module is designed to capture more diverse in-domain patterns that synthesize high-fidelity textures for large corruptions.
arXiv Detail & Related papers (2022-08-25T03:39:24Z) - Learning Prior Feature and Attention Enhanced Image Inpainting [63.21231753407192]
This paper incorporates the pre-training based Masked AutoEncoder (MAE) into the inpainting model.
We propose to use attention priors from MAE to make the inpainting model learn more long-distance dependencies between masked and unmasked regions.
arXiv Detail & Related papers (2022-08-03T04:32:53Z) - Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid [102.24539566851809]
Restoring reasonable and realistic content for arbitrary missing regions in images is an important yet challenging task.
Recent image inpainting models have made significant progress in generating vivid visual details, but they can still lead to texture blurring or structural distortions.
We propose the Semantic Pyramid Network (SPN) motivated by the idea that learning multi-scale semantic priors can greatly benefit the recovery of locally missing content in images.
arXiv Detail & Related papers (2021-12-08T04:33:33Z) - Wavelength-based Attributed Deep Neural Network for Underwater Image
Restoration [9.378355457555319]
This paper shows that attributing the right receptive field size (context) based on the traversing range of the color channel may lead to a substantial performance gain.
As a second novelty, we have incorporated an attentive skip mechanism to adaptively refine the learned multi-contextual features.
The proposed framework, called Deep WaveNet, is optimized using the traditional pixel-wise and feature-based cost functions.
arXiv Detail & Related papers (2021-06-15T06:47:51Z) - TFill: Image Completion via a Transformer-Based Architecture [69.62228639870114]
We propose treating image completion as a directionless sequence-to-sequence prediction task.
We employ a restrictive CNN with small and non-overlapping RF for token representation.
In a second phase, to improve appearance consistency between visible and generated regions, a novel attention-aware layer (AAL) is introduced.
arXiv Detail & Related papers (2021-04-02T01:42:01Z) - Very Long Natural Scenery Image Prediction by Outpainting [96.8509015981031]
Outpainting receives less attention due to two challenges in it.
First challenge is how to keep the spatial and content consistency between generated images and original input.
Second challenge is how to maintain high quality in generated results.
arXiv Detail & Related papers (2019-12-29T16:29:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.