IGR: Improving Diffusion Model for Garment Restoration from Person Image

• URL: http://arxiv.org/abs/2412.11513v1
• Date: Mon, 16 Dec 2024 07:48:30 GMT
• Title: IGR: Improving Diffusion Model for Garment Restoration from Person Image
• Authors: Le Shen, Rong Huang, Zhijie Wang,
• Abstract summary: Garment restoration, the inverse of virtual try-on task, focuses on restoring standard garment from a person image.<n>We propose an improved diffusion model for restoring authentic garments.<n>Our approach employs two garment extractors to independently capture low-level features and high-level semantics from the person image.
• Score: 6.384713545839356
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Garment restoration, the inverse of virtual try-on task, focuses on restoring standard garment from a person image, requiring accurate capture of garment details. However, existing methods often fail to preserve the identity of the garment or rely on complex processes. To address these limitations, we propose an improved diffusion model for restoring authentic garments. Our approach employs two garment extractors to independently capture low-level features and high-level semantics from the person image. Leveraging a pretrained latent diffusion model, these features are integrated into the denoising process through garment fusion blocks, which combine self-attention and cross-attention layers to align the restored garment with the person image. Furthermore, a coarse-to-fine training strategy is introduced to enhance the fidelity and authenticity of the generated garments. Experimental results demonstrate that our model effectively preserves garment identity and generates high-quality restorations, even in challenging scenarios such as complex garments or those with occlusions.

Related papers

• TryOffAnyone: Tiled Cloth Generation from a Dressed Person [1.4732811715354452]
  High-fidelity tiled garment images are essential for personalized recommendations, outfit composition, and virtual try-on systems.<n>We propose a novel approach utilizing a fine-tuned StableDiffusion model.<n>Our method features a streamlined single-stage network design, which integrates garmentspecific masks to isolate and process target clothing items effectively.
  arXiv  Detail & Related papers  (2024-12-11T17:41:53Z)
• AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models [7.534556848810697]
  We propose a novel AnyDressing method for customizing characters conditioned on any combination of garments and personalized text prompts.<n>AnyDressing comprises two primary networks named GarmentsNet and DressingNet, which are respectively dedicated to extracting detailed clothing features.<n>We introduce a Garment-Enhanced Texture Learning strategy to improve the fine-grained texture details of garments.
  arXiv  Detail & Related papers  (2024-12-05T13:16:47Z)
• FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on [73.13242624924814]
  Garment perception enhancement technique, FitDiT, is designed for high-fidelity virtual try-on using Diffusion Transformers (DiT) We introduce a garment texture extractor that incorporates garment priors evolution to fine-tune garment feature, facilitating to better capture rich details such as stripes, patterns, and text. We also employ a dilated-relaxed mask strategy that adapts to the correct length of garments, preventing the generation of garments that fill the entire mask area during cross-category try-on.
  arXiv  Detail & Related papers  (2024-11-15T11:02:23Z)
• Improving Virtual Try-On with Garment-focused Diffusion Models [91.95830983115474]
  Diffusion models have led to the revolutionizing of generative modeling in numerous image synthesis tasks. We shape a new Diffusion model, namely GarDiff, which triggers the garment-focused diffusion process. Experiments on VITON-HD and DressCode datasets demonstrate the superiority of our GarDiff when compared to state-of-the-art VTON approaches.
  arXiv  Detail & Related papers  (2024-09-12T17:55:11Z)
• Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models [69.50286698375386]
  We propose a novel approach that better harnesses diffusion models for face-swapping. We introduce a mask shuffling technique during inpainting training, which allows us to create a so-called universal model for swapping. Ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models.
  arXiv  Detail & Related papers  (2024-09-11T13:43:53Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• AnyFit: Controllable Virtual Try-on for Any Combination of Attire Across Any Scenario [50.62711489896909]
  AnyFit surpasses all baselines on high-resolution benchmarks and real-world data by a large gap. AnyFit's impressive performance on high-fidelity virtual try-ons in any scenario from any image, paves a new path for future research within the fashion community.
  arXiv  Detail & Related papers  (2024-05-28T13:33:08Z)
• ACDG-VTON: Accurate and Contained Diffusion Generation for Virtual Try-On [6.228968332207618]
  We propose a training scheme that limits the scope in which diffusion is trained. Our method runs multi-garment try-on in a single inference cycle and can support high-quality zoomed-in generations without training in higher resolutions.
  arXiv  Detail & Related papers  (2024-03-20T19:45:06Z)
• PFStorer: Personalized Face Restoration and Super-Resolution [19.479263766534345]
  Recent developments in face restoration have achieved remarkable results in producing high-quality and lifelike outputs. The stunning results however often fail to be faithful with respect to the identity of the person as the models lack necessary context. In our approach a restoration model is personalized using a few images of the identity, leading to tailored restoration with respect to the identity while retaining fine-grained details.
  arXiv  Detail & Related papers  (2024-03-13T11:39:30Z)
• Improving Diffusion Models for Authentic Virtual Try-on in the Wild [53.96244595495942]
  This paper considers image-based virtual try-on, which renders an image of a person wearing a curated garment. We propose a novel diffusion model that improves garment fidelity and generates authentic virtual try-on images. We present a customization method using a pair of person-garment images, which significantly improves fidelity and authenticity.
  arXiv  Detail & Related papers  (2024-03-08T08:12:18Z)
• StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On [35.227896906556026]
  Given a clothing image and a person image, an image-based virtual try-on aims to generate a customized image that appears natural and accurately reflects the characteristics of the clothing image. In this work, we aim to expand the applicability of the pre-trained diffusion model so that it can be utilized independently for the virtual try-on task. Our proposed zero cross-attention blocks not only preserve the clothing details by learning the semantic correspondence but also generate high-fidelity images by utilizing the inherent knowledge of the pre-trained model in the warping process.
  arXiv  Detail & Related papers  (2023-12-04T08:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.