An Analysis of Generative Methods for Multiple Image Inpainting

• URL: http://arxiv.org/abs/2205.02146v1
• Date: Wed, 4 May 2022 15:54:08 GMT
• Title: An Analysis of Generative Methods for Multiple Image Inpainting
• Authors: Coloma Ballester, Aurelie Bugeau, Samuel Hurault, Simone Parisotto, Patricia Vitoria
• Abstract summary: Inpainting refers to the restoration of an image with missing regions in a way that is not detectable by the observer. We focus on learning-based image completion methods for multiple and diverse inpainting.
• Score: 4.234843176066354
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image inpainting refers to the restoration of an image with missing regions in a way that is not detectable by the observer. The inpainting regions can be of any size and shape. This is an ill-posed inverse problem that does not have a unique solution. In this work, we focus on learning-based image completion methods for multiple and diverse inpainting which goal is to provide a set of distinct solutions for a given damaged image. These methods capitalize on the probabilistic nature of certain generative models to sample various solutions that coherently restore the missing content. Along the chapter, we will analyze the underlying theory and analyze the recent proposals for multiple inpainting. To investigate the pros and cons of each method, we present quantitative and qualitative comparisons, on common datasets, regarding both the quality and the diversity of the set of inpainted solutions. Our analysis allows us to identify the most successful generative strategies in both inpainting quality and inpainting diversity. This task is closely related to the learning of an accurate probability distribution of images. Depending on the dataset in use, the challenges that entail the training of such a model will be discussed through the analysis.

Related papers

• Trustworthy SR: Resolving Ambiguity in Image Super-resolution via Diffusion Models and Human Feedback [5.665865832321032]
  Super-resolution (SR) is an ill-posed inverse problem with a large set of feasible solutions that are consistent with a given low-resolution image. We propose employing human feedback, where we ask human subjects to select a small number of likely samples and we ensemble the averages of selected samples. Our proposed strategy provides more trustworthy solutions when compared to state-of-the art SR methods.
  arXiv  Detail & Related papers  (2024-02-12T11:55:02Z)
• Learning from small data sets: Patch-based regularizers in inverse problems for image reconstruction [1.1650821883155187]
  Recent advances in machine learning require a huge amount of data and computer capacity to train the networks. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods.
  arXiv  Detail & Related papers  (2023-12-27T15:30:05Z)
• GRIG: Few-Shot Generative Residual Image Inpainting [27.252855062283825]
  We present a novel few-shot generative residual image inpainting method that produces high-quality inpainting results. The core idea is to propose an iterative residual reasoning method that incorporates Convolutional Neural Networks (CNNs) for feature extraction. We also propose a novel forgery-patch adversarial training strategy to create faithful textures and detailed appearances.
  arXiv  Detail & Related papers  (2023-04-24T12:19:06Z)
• Stain-invariant self supervised learning for histopathology image analysis [74.98663573628743]
  We present a self-supervised algorithm for several classification tasks within hematoxylin and eosin stained images of breast cancer. Our method achieves the state-of-the-art performance on several publicly available breast cancer datasets.
  arXiv  Detail & Related papers  (2022-11-14T18:16:36Z)
• Pluralistic Image Completion with Probabilistic Mixture-of-Experts [58.81469985455467]
  We introduce a unified probabilistic graph model that represents the complex interactions in image completion. The entire procedure of image completion is then mathematically divided into several sub-procedures, which helps efficient enforcement of constraints. The inherent parameters of GMM are task-related, which are optimized adaptively during training, while the number of its primitives can control the diversity of results conveniently.
  arXiv  Detail & Related papers  (2022-05-18T17:24:21Z)
• Mining the manifolds of deep generative models for multiple data-consistent solutions of ill-posed tomographic imaging problems [10.115302976900445]
  Tomographic imaging is in general an ill-posed inverse problem. We propose a new empirical sampling method that computes multiple solutions of a tomographic inverse problem.
  arXiv  Detail & Related papers  (2022-02-10T20:27:31Z)
• Generative Probabilistic Image Colorization [2.110198946293069]
  We propose a diffusion-based generative process that trains a sequence of probabilistic models to reverse each step of noise corruption. Given a line-drawing image as input, our method suggests multiple candidate colorized images. Our proposed approach performed well not only on color-conditional image generation tasks, but also on some practical image completion and inpainting tasks.
  arXiv  Detail & Related papers  (2021-09-29T16:10:12Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Invertible Image Rescaling [118.2653765756915]
  We develop an Invertible Rescaling Net (IRN) to produce visually-pleasing low-resolution images. We capture the distribution of the lost information using a latent variable following a specified distribution in the downscaling process.
  arXiv  Detail & Related papers  (2020-05-12T09:55:53Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.