An Inpainting-Infused Pipeline for Attire and Background Replacement
- URL: http://arxiv.org/abs/2402.03501v1
- Date: Mon, 5 Feb 2024 20:34:32 GMT
- Title: An Inpainting-Infused Pipeline for Attire and Background Replacement
- Authors: Felipe Rodrigues Perche-Mahlow and Andr\'e Felipe-Zanella and William
Alberto Cruz-Casta\~neda and Marcellus Amadeus
- Abstract summary: We explore an integrated approach, leveraging advanced techniques in GenAI and computer vision emphasizing image manipulation.
The methodology unfolds through several stages, including depth estimation, the generation and replacement of backgrounds.
Experiments conducted in this study underscore the methodology's efficacy, highlighting its potential to produce visually captivating content.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In recent years, groundbreaking advancements in Generative Artificial
Intelligence (GenAI) have triggered a transformative paradigm shift,
significantly influencing various domains. In this work, we specifically
explore an integrated approach, leveraging advanced techniques in GenAI and
computer vision emphasizing image manipulation. The methodology unfolds through
several stages, including depth estimation, the creation of inpaint masks based
on depth information, the generation and replacement of backgrounds utilizing
Stable Diffusion in conjunction with Latent Consistency Models (LCMs), and the
subsequent replacement of clothes and application of aesthetic changes through
an inpainting pipeline. Experiments conducted in this study underscore the
methodology's efficacy, highlighting its potential to produce visually
captivating content. The convergence of these advanced techniques allows users
to input photographs of individuals and manipulate them to modify clothing and
background based on specific prompts without manually input inpainting masks,
effectively placing the subjects within the vast landscape of creative
imagination.
Related papers
- TALE: Training-free Cross-domain Image Composition via Adaptive Latent Manipulation and Energy-guided Optimization [59.412236435627094]
TALE is a training-free framework harnessing the generative capabilities of text-to-image diffusion models.
We equip TALE with two mechanisms dubbed Adaptive Latent Manipulation and Energy-guided Latent Optimization.
Our experiments demonstrate that TALE surpasses prior baselines and attains state-of-the-art performance in image-guided composition.
arXiv Detail & Related papers (2024-08-07T08:52:21Z) - Sketch-guided Image Inpainting with Partial Discrete Diffusion Process [5.005162730122933]
We introduce a novel partial discrete diffusion process (PDDP) for sketch-guided inpainting.
PDDP corrupts the masked regions of the image and reconstructs these masked regions conditioned on hand-drawn sketches.
The proposed novel transformer module accepts two inputs -- the image containing the masked region to be inpainted and the query sketch to model the reverse diffusion process.
arXiv Detail & Related papers (2024-04-18T07:07:38Z) - BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed
Dual-Branch Diffusion [61.90969199199739]
BrushNet is a novel plug-and-play dual-branch model engineered to embed pixel-level masked image features into any pre-trained DM.
BrushNet's superior performance over existing models across seven key metrics, including image quality, mask region preservation, and textual coherence.
arXiv Detail & Related papers (2024-03-11T17:59:31Z) - Disentangled Representation Learning for Controllable Person Image
Generation [29.719070087384512]
We propose a novel framework named DRL-CPG to learn disentangled latent representation for controllable person image generation.
To our knowledge, we are the first to learn disentangled latent representations with transformers for person image generation.
arXiv Detail & Related papers (2023-12-10T07:15:58Z) - Masking Improves Contrastive Self-Supervised Learning for ConvNets, and Saliency Tells You Where [63.61248884015162]
We aim to alleviate the burden of including masking operation into the contrastive-learning framework for convolutional neural networks.
We propose to explicitly take the saliency constraint into consideration in which the masked regions are more evenly distributed among the foreground and background.
arXiv Detail & Related papers (2023-09-22T09:58:38Z) - Deep Image Matting: A Comprehensive Survey [85.77905619102802]
This paper presents a review of recent advancements in image matting in the era of deep learning.
We focus on two fundamental sub-tasks: auxiliary input-based image matting and automatic image matting.
We discuss relevant applications of image matting and highlight existing challenges and potential opportunities for future research.
arXiv Detail & Related papers (2023-04-10T15:48:55Z) - Expanding the Latent Space of StyleGAN for Real Face Editing [4.1715767752637145]
A surge of face editing techniques have been proposed to employ the pretrained StyleGAN for semantic manipulation.
To successfully edit a real image, one must first convert the input image into StyleGAN's latent variables.
We present a method to expand the latent space of StyleGAN with additional content features to break down the trade-off between low-distortion and high-editability.
arXiv Detail & Related papers (2022-04-26T18:27:53Z) - MAT: Mask-Aware Transformer for Large Hole Image Inpainting [79.67039090195527]
We present a novel model for large hole inpainting, which unifies the merits of transformers and convolutions.
Experiments demonstrate the state-of-the-art performance of the new model on multiple benchmark datasets.
arXiv Detail & Related papers (2022-03-29T06:36:17Z) - Look here! A parametric learning based approach to redirect visual
attention [49.609412873346386]
We introduce an automatic method to make an image region more attention-capturing via subtle image edits.
Our model predicts a distinct set of global parametric transformations to be applied to the foreground and background image regions.
Our edits enable inference at interactive rates on any image size, and easily generalize to videos.
arXiv Detail & Related papers (2020-08-12T16:08:36Z) - Interactive Neural Style Transfer with Artists [6.130486652666935]
We present interactive painting processes in which a painter and various neural style transfer algorithms interact on a real canvas.
We gather a set of paired painting-pictures images and present a new evaluation methodology based on the predictivity of neural style transfer algorithms.
arXiv Detail & Related papers (2020-03-14T15:27:44Z) - Learning Transformation-Aware Embeddings for Image Forensics [15.484408315588569]
Image Provenance Analysis aims at discovering relationships among different manipulated image versions that share content.
One of the main sub-problems for provenance analysis that has not yet been addressed directly is the edit ordering of images that share full content or are near-duplicates.
This paper introduces a novel deep learning-based approach to provide a plausible ordering to images that have been generated from a single image through transformations.
arXiv Detail & Related papers (2020-01-13T22:01:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.