Recursive InPainting (RIP): how much information is lost under recursive inferences?

• URL: http://arxiv.org/abs/2407.09549v2
• Date: Sun, 25 May 2025 14:57:09 GMT
• Title: Recursive InPainting (RIP): how much information is lost under recursive inferences?
• Authors: Javier Conde, Miguel González, Gonzalo Martínez, Fernando Moral, Elena Merino-Gómez, Pedro Reviriego,
• Abstract summary: The rapid adoption of generative artificial intelligence is accelerating content creation and modification.<n>This poses new risks; for example, AI-generated content may be used to train newer AI models and degrade their performance.<n>An example of AI image modifications is inpainting in which an AI model completes missing fragments of an image.
• Score: 40.65612212208553
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid adoption of generative artificial intelligence (AI) is accelerating content creation and modification. For example, variations of a given content, be it text or images, can be created almost instantly and at a low cost. This will soon lead to the majority of text and images being created directly by AI models or by humans assisted by AI. This poses new risks; for example, AI-generated content may be used to train newer AI models and degrade their performance, or information may be lost in the transformations made by AI which could occur when the same content is processed over and over again by AI tools. An example of AI image modifications is inpainting in which an AI model completes missing fragments of an image. The incorporation of inpainting tools into photo editing programs promotes their adoption and encourages their recursive use to modify images. Inpainting can be applied recursively, starting from an image, removing some parts, applying inpainting to reconstruct the image, revising it, and then starting the inpainting process again on the reconstructed image, etc. This paper presents an empirical evaluation of recursive inpainting when using one of the most widely used image models: Stable Diffusion. The inpainting process is applied by randomly selecting a fragment of the image, reconstructing it, selecting another fragment, and repeating the process a predefined number of iterations. The images used in the experiments are taken from a publicly available art data set and correspond to different styles and historical periods. Additionally, photographs are also evaluated as a reference. The modified images are compared with the original ones by both using quantitative metrics and performing a qualitative analysis. The results show that recursive inpainting in some cases modifies the image so that it still resembles the original one while in others leads to degeneration.

Related papers

• SAGI: Semantically Aligned and Uncertainty Guided AI Image Inpainting [11.216906046169683]
  SAGI-D is the largest and most diverse dataset of AI-generated inpaintings.<n>Our experiments show that semantic alignment significantly improves image quality and aesthetics.<n>Using SAGI-D for training several image forensic approaches increases in-domain detection performance on average by 37.4%.
  arXiv  Detail & Related papers  (2025-02-10T15:56:28Z)
• A Sanity Check for AI-generated Image Detection [49.08585395873425]
  We propose AIDE (AI-generated Image DEtector with Hybrid Features) to detect AI-generated images.<n>AIDE achieves +3.5% and +4.6% improvements to state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-06-27T17:59:49Z)
• Fill in the ____ (a Diffusion-based Image Inpainting Pipeline) [0.0]
  Inpainting is the process of taking an image and generating lost or intentionally occluded portions. Modern inpainting techniques have shown remarkable ability in generating sensible completions. A critical gap in these existing models will be addressed, focusing on the ability to prompt and control what exactly is generated.
  arXiv  Detail & Related papers  (2024-03-24T05:26:55Z)
• Online Detection of AI-Generated Images [17.30253784649635]
  We study generalization in this setting, training on N models and testing on the next (N+k) We extend this approach to pixel prediction, demonstrating strong performance using automatically-generated inpainted data. In addition, for settings where commercial models are not publicly available for automatic data generation, we evaluate if pixel detectors can be trained solely on whole synthetic images.
  arXiv  Detail & Related papers  (2023-10-23T17:53:14Z)
• Evaluating Data Attribution for Text-to-Image Models [62.844382063780365]
  We evaluate attribution through "customization" methods, which tune an existing large-scale model toward a given exemplar object or style. Our key insight is that this allows us to efficiently create synthetic images that are computationally influenced by the exemplar by construction. By taking into account the inherent uncertainty of the problem, we can assign soft attribution scores over a set of training images.
  arXiv  Detail & Related papers  (2023-06-15T17:59:51Z)
• Inst-Inpaint: Instructing to Remove Objects with Diffusion Models [18.30057229657246]
  In this work, we are interested in an image inpainting algorithm that estimates which object to be removed based on natural language input and removes it, simultaneously. We present a novel inpainting framework, Inst-Inpaint, that can remove objects from images based on the instructions given as text prompts.
  arXiv  Detail & Related papers  (2023-04-06T17:29:50Z)
• Invertible Rescaling Network and Its Extensions [118.72015270085535]
  In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective. We develop invertible models to generate valid degraded images and transform the distribution of lost contents. Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
  arXiv  Detail & Related papers  (2022-10-09T06:58:58Z)
• Re-Imagen: Retrieval-Augmented Text-to-Image Generator [58.60472701831404]
  Retrieval-Augmented Text-to-Image Generator (Re-Imagen) Retrieval-Augmented Text-to-Image Generator (Re-Imagen)
  arXiv  Detail & Related papers  (2022-09-29T00:57:28Z)
• Perceptual Artifacts Localization for Inpainting [60.5659086595901]
  We propose a new learning task of automatic segmentation of inpainting perceptual artifacts. We train advanced segmentation networks on a dataset to reliably localize inpainting artifacts within inpainted images. We also propose a new evaluation metric called Perceptual Artifact Ratio (PAR), which is the ratio of objectionable inpainted regions to the entire inpainted area.
  arXiv  Detail & Related papers  (2022-08-05T18:50:51Z)
• Restore from Restored: Single-image Inpainting [9.699531255678856]
  We present a novel and efficient self-supervised fine-tuning algorithm for inpainting networks. We update the parameters of the pre-trained inpainting networks by utilizing existing self-similar patches. We achieve state-of-the-art inpainting results on publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-25T11:38:51Z)
• Ensembling with Deep Generative Views [72.70801582346344]
  generative models can synthesize "views" of artificial images that mimic real-world variations, such as changes in color or pose. Here, we investigate whether such views can be applied to real images to benefit downstream analysis tasks such as image classification. We use StyleGAN2 as the source of generative augmentations and investigate this setup on classification tasks involving facial attributes, cat faces, and cars.
  arXiv  Detail & Related papers  (2021-04-29T17:58:35Z)
• In&Out : Diverse Image Outpainting via GAN Inversion [89.84841983778672]
  Image outpainting seeks for a semantically consistent extension of the input image beyond its available content. In this work, we formulate the problem from the perspective of inverting generative adversarial networks. Our generator renders micro-patches conditioned on their joint latent code as well as their individual positions in the image.
  arXiv  Detail & Related papers  (2021-04-01T17:59:10Z)
• Restore from Restored: Single-image Inpainting [9.699531255678856]
  We present a novel and efficient self-supervised fine-tuning algorithm for inpainting networks. We upgrade the parameters of the pretrained networks by utilizing existing self-similar patches within the given input image. We achieve state-of-the-art inpainting results on publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2021-02-16T10:59:28Z)
• High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling [122.06593036862611]
  Existing image inpainting methods often produce artifacts when dealing with large holes in real applications. We propose an iterative inpainting method with a feedback mechanism. Experiments show that our method significantly outperforms existing methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2020-05-24T13:23:45Z)
• Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation [181.08127307338654]
  This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
  arXiv  Detail & Related papers  (2020-03-30T17:45:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.