R-MNet: A Perceptual Adversarial Network for Image Inpainting
- URL: http://arxiv.org/abs/2008.04621v3
- Date: Mon, 9 Nov 2020 17:08:51 GMT
- Title: R-MNet: A Perceptual Adversarial Network for Image Inpainting
- Authors: Jireh Jam and Connah Kendrick and Vincent Drouard and Kevin Walker and
Gee-Sern Hsu and Moi Hoon Yap
- Abstract summary: We propose a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting.
We show that our method is able to generalize to high-resolution inpainting task, and further show more realistic outputs that are plausible to the human visual system.
- Score: 5.471225956329675
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Facial image inpainting is a problem that is widely studied, and in recent
years the introduction of Generative Adversarial Networks, has led to
improvements in the field. Unfortunately some issues persists, in particular
when blending the missing pixels with the visible ones. We address the problem
by proposing a Wasserstein GAN combined with a new reverse mask operator,
namely Reverse Masking Network (R-MNet), a perceptual adversarial network for
image inpainting. The reverse mask operator transfers the reverse masked image
to the end of the encoder-decoder network leaving only valid pixels to be
inpainted. Additionally, we propose a new loss function computed in feature
space to target only valid pixels combined with adversarial training. These
then capture data distributions and generate images similar to those in the
training data with achieved realism (realistic and coherent) on the output
images. We evaluate our method on publicly available dataset, and compare with
state-of-the-art methods. We show that our method is able to generalize to
high-resolution inpainting task, and further show more realistic outputs that
are plausible to the human visual system when compared with the
state-of-the-art methods.
Related papers
- T-former: An Efficient Transformer for Image Inpainting [50.43302925662507]
A class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields.
In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion, and based on this attention, a network called $T$-former is designed for image inpainting.
Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity.
arXiv Detail & Related papers (2023-05-12T04:10:42Z) - What's Behind the Mask: Estimating Uncertainty in Image-to-Image
Problems [29.55144604924458]
Estimating uncertainty in image-to-image networks is an important task.
We introduce a new approach to this problem based on masking.
We evaluate our mask-based approach to uncertainty on image colorization, image completion, and super-resolution tasks.
arXiv Detail & Related papers (2022-11-28T10:41:34Z) - Contextual Attention Mechanism, SRGAN Based Inpainting System for
Eliminating Interruptions from Images [2.894944733573589]
We propose an end-to-end pipeline for inpainting images using a complete Machine Learning approach.
We first use the YOLO model to automatically identify and localize the object we wish to remove from the image.
After this, we provide the masked image and original image to the GAN model which uses the Contextual Attention method to fill in the region.
arXiv Detail & Related papers (2022-04-06T05:51:04Z) - A Wasserstein GAN for Joint Learning of Inpainting and its Spatial
Optimisation [3.4392739159262145]
We propose the first generative adversarial network for spatial inpainting data optimisation.
In contrast to previous approaches, it allows joint training of an inpainting generator and a corresponding mask network.
This yields significant improvements in visual quality and speed over conventional models and also outperforms current optimisation networks.
arXiv Detail & Related papers (2022-02-11T14:02:36Z) - RePaint: Inpainting using Denoising Diffusion Probabilistic Models [161.74792336127345]
Free-form inpainting is the task of adding new content to an image in the regions specified by an arbitrary binary mask.
We propose RePaint: A Denoising Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks.
We validate our method for both faces and general-purpose image inpainting using standard and extreme masks.
arXiv Detail & Related papers (2022-01-24T18:40:15Z) - Inverse Problems Leveraging Pre-trained Contrastive Representations [88.70821497369785]
We study a new family of inverse problems for recovering representations of corrupted data.
We propose a supervised inversion method that uses a contrastive objective to obtain excellent representations for highly corrupted images.
Our method outperforms end-to-end baselines even with a fraction of the labeled data in a wide range of forward operators.
arXiv Detail & Related papers (2021-10-14T15:06:30Z) - Free-Form Image Inpainting via Contrastive Attention Network [64.05544199212831]
In image inpainting tasks, masks with any shapes can appear anywhere in images which form complex patterns.
It is difficult for encoders to capture such powerful representations under this complex situation.
We propose a self-supervised Siamese inference network to improve the robustness and generalization.
arXiv Detail & Related papers (2020-10-29T14:46:05Z) - High-Resolution Image Inpainting with Iterative Confidence Feedback and
Guided Upsampling [122.06593036862611]
Existing image inpainting methods often produce artifacts when dealing with large holes in real applications.
We propose an iterative inpainting method with a feedback mechanism.
Experiments show that our method significantly outperforms existing methods in both quantitative and qualitative evaluations.
arXiv Detail & Related papers (2020-05-24T13:23:45Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z) - Symmetric Skip Connection Wasserstein GAN for High-Resolution Facial
Image Inpainting [5.163405308079487]
We propose a Symmetric Skip Connection Wasserstein Generative Adversarial Network (S-WGAN) for high-resolution facial image inpainting.
The architecture is an encoder-decoder with convolutional blocks, linked by skip connections.
We evaluate our method and the state-of-the-art methods on CelebA-HQ dataset.
arXiv Detail & Related papers (2020-01-11T09:09:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.