Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images

• URL: http://arxiv.org/abs/2204.02591v1
• Date: Wed, 6 Apr 2022 05:51:04 GMT
• Title: Contextual Attention Mechanism, SRGAN Based Inpainting System for Eliminating Interruptions from Images
• Authors: Narayana Darapaneni, Vaibhav Kherde, Kameswara Rao, Deepali Nikam, Swanand Katdare, Anima Shukla, Anagha Lomate, Anwesh Reddy Paduri
• Abstract summary: We propose an end-to-end pipeline for inpainting images using a complete Machine Learning approach. We first use the YOLO model to automatically identify and localize the object we wish to remove from the image. After this, we provide the masked image and original image to the GAN model which uses the Contextual Attention method to fill in the region.
• Score: 2.894944733573589
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The new alternative is to use deep learning to inpaint any image by utilizing image classification and computer vision techniques. In general, image inpainting is a task of recreating or reconstructing any broken image which could be a photograph or oil/acrylic painting. With the advancement in the field of Artificial Intelligence, this topic has become popular among AI enthusiasts. With our approach, we propose an initial end-to-end pipeline for inpainting images using a complete Machine Learning approach instead of a conventional application-based approach. We first use the YOLO model to automatically identify and localize the object we wish to remove from the image. Using the result obtained from the model we can generate a mask for the same. After this, we provide the masked image and original image to the GAN model which uses the Contextual Attention method to fill in the region. It consists of two generator networks and two discriminator networks and is also called a coarse-to-fine network structure. The two generators use fully convolutional networks while the global discriminator gets hold of the entire image as input while the local discriminator gets the grip of the filled region as input. The contextual Attention mechanism is proposed to effectively borrow the neighbor information from distant spatial locations for reconstructing the missing pixels. The third part of our implementation uses SRGAN to resolve the inpainted image back to its original size. Our work is inspired by the paper Free-Form Image Inpainting with Gated Convolution and Generative Image Inpainting with Contextual Attention.

Related papers

• Efficient and Explicit Modelling of Image Hierarchies for Image Restoration [120.35246456398738]
  We propose a mechanism to efficiently and explicitly model image hierarchies in the global, regional, and local range for image restoration. Inspired by that, we propose the anchored stripe self-attention which achieves a good balance between the space and time complexity of self-attention. Then we propose a new network architecture dubbed GRL to explicitly model image hierarchies in the Global, Regional, and Local range.
  arXiv  Detail & Related papers  (2023-03-01T18:59:29Z)
• Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid [102.24539566851809]
  Restoring reasonable and realistic content for arbitrary missing regions in images is an important yet challenging task. Recent image inpainting models have made significant progress in generating vivid visual details, but they can still lead to texture blurring or structural distortions. We propose the Semantic Pyramid Network (SPN) motivated by the idea that learning multi-scale semantic priors can greatly benefit the recovery of locally missing content in images.
  arXiv  Detail & Related papers  (2021-12-08T04:33:33Z)
• SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches [95.45728042499836]
  We propose a new paradigm of sketch-based image manipulation: mask-free local image manipulation. Our model automatically predicts the target modification region and encodes it into a structure style vector. A generator then synthesizes the new image content based on the style vector and sketch.
  arXiv  Detail & Related papers  (2021-11-30T02:42:31Z)
• JPGNet: Joint Predictive Filtering and Generative Network for Image Inpainting [21.936689731138213]
  Image inpainting aims to restore the missing regions and make the recovery results identical to the originally complete image. Existing works usually regard it as a pure generation problem and employ cutting-edge generative techniques to address it. In this paper, we formulate image inpainting as a mix of two problems, i.e., predictive filtering and deep generation.
  arXiv  Detail & Related papers  (2021-07-09T07:49:52Z)
• Iterative Facial Image Inpainting using Cyclic Reverse Generator [0.913755431537592]
  Cyclic Reverse Generator (CRG) architecture provides an encoder-generator model. We empirically observed that only a few iterations are sufficient to generate realistic images with the proposed model. Our method allows applying sketch-based inpaintings, using variety of mask types, and producing multiple and diverse results.
  arXiv  Detail & Related papers  (2021-01-18T12:19:58Z)
• Free-Form Image Inpainting via Contrastive Attention Network [64.05544199212831]
  In image inpainting tasks, masks with any shapes can appear anywhere in images which form complex patterns. It is difficult for encoders to capture such powerful representations under this complex situation. We propose a self-supervised Siamese inference network to improve the robustness and generalization.
  arXiv  Detail & Related papers  (2020-10-29T14:46:05Z)
• Deep Generative Model for Image Inpainting with Local Binary Pattern Learning and Spatial Attention [28.807711307545112]
  We propose a new end-to-end, two-stage (coarse-to-fine) generative model through combining a local binary pattern (LBP) learning network with an actual inpainting network. Experiments on public datasets including CelebA-HQ, Places and Paris StreetView demonstrate that our model generates better inpainting results than the state-of-the-art competing algorithms.
  arXiv  Detail & Related papers  (2020-09-02T12:59:28Z)
• R-MNet: A Perceptual Adversarial Network for Image Inpainting [5.471225956329675]
  We propose a Wasserstein GAN combined with a new reverse mask operator, namely Reverse Masking Network (R-MNet), a perceptual adversarial network for image inpainting. We show that our method is able to generalize to high-resolution inpainting task, and further show more realistic outputs that are plausible to the human visual system.
  arXiv  Detail & Related papers  (2020-08-11T10:58:10Z)
• A U-Net Based Discriminator for Generative Adversarial Networks [86.67102929147592]
  We propose an alternative U-Net based discriminator architecture for generative adversarial networks (GANs) The proposed architecture allows to provide detailed per-pixel feedback to the generator while maintaining the global coherence of synthesized images. The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics.
  arXiv  Detail & Related papers  (2020-02-28T11:16:54Z)
• OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clustering [100.32273175423146]
  We present a method for simultaneously learning, in an unsupervised manner, a conditional image generator, foreground extraction and segmentation, and object removal and background completion. The method combines a Geneversarative Adrial Network and a Variational Auto-Encoder, with multiple encoders, generators and discriminators, and benefits from solving all tasks at once.
  arXiv  Detail & Related papers  (2019-12-31T18:15:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.