Pixel-wise Dense Detector for Image Inpainting
- URL: http://arxiv.org/abs/2011.02293v2
- Date: Tue, 17 Nov 2020 09:27:30 GMT
- Title: Pixel-wise Dense Detector for Image Inpainting
- Authors: Ruisong Zhang, Weize Quan, Baoyuan Wu, Zhifeng Li, Dong-Ming Yan
- Abstract summary: Recent GAN-based image inpainting approaches adopt an average strategy to discriminate the generated image and output a scalar.
We propose a novel detection-based generative framework for image inpainting, which adopts the min-max strategy in an adversarial process.
Experiments on multiple public datasets show the superior performance of the proposed framework.
- Score: 34.721991959357425
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent GAN-based image inpainting approaches adopt an average strategy to
discriminate the generated image and output a scalar, which inevitably lose the
position information of visual artifacts. Moreover, the adversarial loss and
reconstruction loss (e.g., l1 loss) are combined with tradeoff weights, which
are also difficult to tune. In this paper, we propose a novel detection-based
generative framework for image inpainting, which adopts the min-max strategy in
an adversarial process. The generator follows an encoder-decoder architecture
to fill the missing regions, and the detector using weakly supervised learning
localizes the position of artifacts in a pixel-wise manner. Such position
information makes the generator pay attention to artifacts and further enhance
them. More importantly, we explicitly insert the output of the detector into
the reconstruction loss with a weighting criterion, which balances the weight
of the adversarial loss and reconstruction loss automatically rather than
manual operation. Experiments on multiple public datasets show the superior
performance of the proposed framework. The source code is available at
https://github.com/Evergrow/GDN_Inpainting.
Related papers
- Panoramic Image Inpainting With Gated Convolution And Contextual
Reconstruction Loss [19.659176149635417]
We propose a panoramic image inpainting framework that consists of a Face Generator, a Cube Generator, a side branch, and two discriminators.
The proposed method is compared with state-of-the-art (SOTA) methods on SUN360 Street View dataset in terms of PSNR and SSIM.
arXiv Detail & Related papers (2024-02-05T11:58:08Z) - Inpainting Normal Maps for Lightstage data [3.1002416427168304]
This study introduces a novel method for inpainting normal maps using a generative adversarial network (GAN)
Our approach extends previous general image inpainting techniques, employing a bow tie-like generator network and a discriminator network, with alternating training phases.
Our findings suggest that the proposed model effectively generates high-quality, realistic inpainted normal maps, suitable for performance capture applications.
arXiv Detail & Related papers (2024-01-16T03:59:07Z) - AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation [51.143540967290114]
We propose a method that unlocks a wide range of previously-infeasible geometric augmentations for unsupervised depth computation and estimation.
This is achieved by reversing, or undo''-ing, geometric transformations to the coordinates of the output depth, warping the depth map back to the original reference frame.
arXiv Detail & Related papers (2023-10-15T05:15:45Z) - Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
Digital image forensics plays a crucial role in image authentication and manipulation localization.
This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts.
Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
arXiv Detail & Related papers (2023-09-30T02:54:51Z) - RGI: robust GAN-inversion for mask-free image inpainting and
unsupervised pixel-wise anomaly detection [18.10039647382319]
We propose a Robust GAN-inversion (RGI) method with a provable robustness guarantee to achieve image restoration under unknown textitgross corruptions.
We show that the restored image and the identified corrupted region mask convergeally to the ground truth.
The proposed RGI/R-RGI method unifies two important applications with state-of-the-art (SOTA) performance.
arXiv Detail & Related papers (2023-02-24T05:43:03Z) - Inpainting borehole images using Generative Adversarial Networks [0.0]
We propose a GAN-based approach for gap filling in borehole images created by wireline microresistivity imaging tools.
The proposed method utilizes a generator, global discriminator, and local discriminator to inpaint the missing regions of the image.
arXiv Detail & Related papers (2023-01-15T18:15:52Z) - RestoreDet: Degradation Equivariant Representation for Object Detection
in Low Resolution Images [81.91416537019835]
We propose a novel framework, RestoreDet, to detect objects in degraded low resolution images.
Our framework based on CenterNet has achieved superior performance compared with existing methods when facing variant degradation situations.
arXiv Detail & Related papers (2022-01-07T03:40:23Z) - Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
We present a learning-based solution for restoring images suffering from spatially-varying degradations.
We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
arXiv Detail & Related papers (2021-08-19T11:02:25Z) - Exploiting Deep Generative Prior for Versatile Image Restoration and
Manipulation [181.08127307338654]
This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images.
The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
arXiv Detail & Related papers (2020-03-30T17:45:07Z) - Image Fine-grained Inpainting [89.17316318927621]
We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields.
To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss.
We also employ a discriminator with local and global branches to ensure local-global contents consistency.
arXiv Detail & Related papers (2020-02-07T03:45:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.