Adaptive Image Inpainting

• URL: http://arxiv.org/abs/2201.00177v1
• Date: Sat, 1 Jan 2022 12:16:01 GMT
• Title: Adaptive Image Inpainting
• Authors: Maitreya Suin, Kuldeep Purohit, A. N. Rajagopalan
• Abstract summary: Inpainting methods have shown significant improvements by using deep neural networks. The problem is rooted in the encoder layers' ineffectiveness in building a complete and faithful embedding of the missing regions. We propose a distillation based approach for inpainting, where we provide direct feature level supervision for the encoder layers.
• Score: 43.02281823557039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image inpainting methods have shown significant improvements by using deep neural networks recently. However, many of these techniques often create distorted structures or blurry textures inconsistent with surrounding areas. The problem is rooted in the encoder layers' ineffectiveness in building a complete and faithful embedding of the missing regions. To address this problem, two-stage approaches deploy two separate networks for a coarse and fine estimate of the inpainted image. Some approaches utilize handcrafted features like edges or contours to guide the reconstruction process. These methods suffer from huge computational overheads owing to multiple generator networks, limited ability of handcrafted features, and sub-optimal utilization of the information present in the ground truth. Motivated by these observations, we propose a distillation based approach for inpainting, where we provide direct feature level supervision for the encoder layers in an adaptive manner. We deploy cross and self distillation techniques and discuss the need for a dedicated completion-block in encoder to achieve the distillation target. We conduct extensive evaluations on multiple datasets to validate our method.

Related papers

• Dense Feature Interaction Network for Image Inpainting Localization [28.028361409524457]
  Inpainting can be used to conceal or alter image contents in malicious manipulation of images. Existing methods mostly rely on a basic encoder-decoder structure, which often results in a high number of false positives. In this paper, we describe a new method for inpainting detection based on a Dense Feature Interaction Network (DeFI-Net)
  arXiv  Detail & Related papers  (2024-08-05T02:35:13Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Two approaches to inpainting microstructure with deep convolutional generative adversarial networks [0.0]
  Microstructural inpainting is a method to replace occluded regions with synthetic microstructure with matching boundaries. In this paper we introduce two methods that use generative adversarial networks to generate contiguous inpainted regions of arbitrary shape and size. We find that one benefits from high speed and simplicity, whilst the other gives smoother boundaries at the inpainting border.
  arXiv  Detail & Related papers  (2022-10-13T13:08:24Z)
• Unbiased Multi-Modality Guidance for Image Inpainting [27.286351511243502]
  We develop an end-to-end multi-modality guided transformer network for image inpainting. Within each transformer block, the proposed spatial-aware attention module can learn the multi-modal structural features efficiently. Our method enriches semantically consistent context in an image based on discriminative information from multiple modalities.
  arXiv  Detail & Related papers  (2022-08-25T03:13:43Z)
• Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation [111.89519571205778]
  In this work, we propose an alternative domain-adaptive approach to depth estimation. Our novel two-step structure first trains a depth estimation network with labeled synthetic images in a supervised manner. The results of our experiments show that the proposed method improves the network's performance on real images by a considerable margin.
  arXiv  Detail & Related papers  (2021-09-24T08:11:34Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• Semantic Layout Manipulation with High-Resolution Sparse Attention [106.59650698907953]
  We tackle the problem of semantic image layout manipulation, which aims to manipulate an input image by editing its semantic label map. A core problem of this task is how to transfer visual details from the input images to the new semantic layout while making the resulting image visually realistic. We propose a high-resolution sparse attention module that effectively transfers visual details to new layouts at a resolution up to 512x512.
  arXiv  Detail & Related papers  (2020-12-14T06:50:43Z)
• Texture Transform Attention for Realistic Image Inpainting [6.275013056564918]
  We propose a Texture Transform Attention network that better produces the missing region inpainting with fine details. Texture Transform Attention is used to create a new reassembled texture map using fine textures and coarse semantics. We evaluate our model end-to-end with the publicly available datasets CelebA-HQ and Places2.
  arXiv  Detail & Related papers  (2020-12-08T06:28:51Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.