VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization

• URL: http://arxiv.org/abs/2103.16874v1
• Date: Wed, 31 Mar 2021 07:52:41 GMT
• Title: VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization
• Authors: Seunghwan Choi, Sunghyun Park, Minsoo Lee, Jaegul Choo
• Abstract summary: We propose a novel virtual try-on method called VITON-HD that successfully synthesizes 1024x768 virtual try-on images. We show that VITON-HD highly sur-passes the baselines in terms of synthesized image quality both qualitatively and quantitatively.
• Score: 18.347532903864597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of image-based virtual try-on aims to transfer a target clothing item onto the corresponding region of a person, which is commonly tackled by fitting the item to the desired body part and fusing the warped item with the person. While an increasing number of studies have been conducted, the resolution of synthesized images is still limited to low (e.g., 256x192), which acts as the critical limitation against satisfying online consumers. We argue that the limitation stems from several challenges: as the resolution increases, the artifacts in the misaligned areas between the warped clothes and the desired clothing regions become noticeable in the final results; the architectures used in existing methods have low performance in generating high-quality body parts and maintaining the texture sharpness of the clothes. To address the challenges, we propose a novel virtual try-on method called VITON-HD that successfully synthesizes 1024x768 virtual try-on images. Specifically, we first prepare the segmentation map to guide our virtual try-on synthesis, and then roughly fit the target clothing item to a given person's body. Next, we propose ALIgnment-Aware Segment (ALIAS) normalization and ALIAS generator to handle the misaligned areas and preserve the details of 1024x768 inputs. Through rigorous comparison with existing methods, we demonstrate that VITON-HD highly sur-passes the baselines in terms of synthesized image quality both qualitatively and quantitatively.

Related papers

• Time-Efficient and Identity-Consistent Virtual Try-On Using A Variant of Altered Diffusion Models [4.038493506169702]
  This study emphasizes the challenges of preserving intricate texture details and distinctive features of the target person and the clothes in various scenarios. Various existing approaches are explored, highlighting the limitations and unresolved aspects. It then proposes a novel diffusion-based solution that addresses garment texture preservation and user identity retention during virtual try-on.
  arXiv  Detail & Related papers  (2024-03-12T07:15:29Z)
• WarpDiffusion: Efficient Diffusion Model for High-Fidelity Virtual Try-on [81.15988741258683]
  Image-based Virtual Try-On (VITON) aims to transfer an in-shop garment image onto a target person. Current methods often overlook the synthesis quality around the garment-skin boundary and realistic effects like wrinkles and shadows on the warped garments. We propose WarpDiffusion, which bridges the warping-based and diffusion-based paradigms via a novel informative and local garment feature attention mechanism.
  arXiv  Detail & Related papers  (2023-12-06T18:34:32Z)
• VTON-IT: Virtual Try-On using Image Translation [0.0]
  We try to produce photo-realistic translated images through semantic segmentation and a generative adversarial architecture-based image translation network. We present a novel image-based Virtual Try-On application VTON-IT that takes an RGB image, segments desired body part, and overlays target cloth over the segmented body region.
  arXiv  Detail & Related papers  (2023-10-06T19:47:20Z)
• OccluMix: Towards De-Occlusion Virtual Try-on by Semantically-Guided Mixup [79.3118064406151]
  Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes) Prior methods successfully preserve the character of clothing images. Occlusion remains a pernicious effect for realistic virtual try-on.
  arXiv  Detail & Related papers  (2023-01-03T06:29:11Z)
• High-Resolution Virtual Try-On with Misalignment and Occlusion-Handled Conditions [29.236895355922496]
  Image-based virtual try-on aims to synthesize an image of a person wearing a given clothing item. We propose a novel try-on condition generator as a unified module of the two stages (i.e., warping and segmentation generation stages) A newly proposed feature fusion block in the condition generator implements the information exchange, and the condition generator does not create any misalignment or pixel-squeezing artifacts.
  arXiv  Detail & Related papers  (2022-06-28T17:47:53Z)
• Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN [66.3650689395967]
  We propose a texture-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN (PASTA-GAN), that facilitates real-world unpaired virtual try-on. To disentangle the style and spatial information of each garment, PASTA-GAN consists of an innovative patch-routed disentanglement module.
  arXiv  Detail & Related papers  (2021-11-20T08:36:12Z)
• Data Augmentation using Random Image Cropping for High-resolution Virtual Try-On (VITON-CROP) [18.347532903864597]
  VITON-CROP synthesizes images more robustly when integrated with random crop augmentation compared to the existing state-of-the-art virtual try-on models. In the experiments, we demonstrate that VITON-CROP is superior to VITON-HD both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2021-11-16T07:40:16Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• Cloth Interactive Transformer for Virtual Try-On [106.21605249649957]
  We propose a novel two-stage cloth interactive transformer (CIT) method for the virtual try-on task. In the first stage, we design a CIT matching block, aiming to precisely capture the long-range correlations between the cloth-agnostic person information and the in-shop cloth information. In the second stage, we put forth a CIT reasoning block for establishing global mutual interactive dependencies among person representation, the warped clothing item, and the corresponding warped cloth mask.
  arXiv  Detail & Related papers  (2021-04-12T14:45:32Z)
• Aggregated Contextual Transformations for High-Resolution Image Inpainting [57.241749273816374]
  We propose an enhanced GAN-based model, named Aggregated COntextual-Transformation GAN (AOT-GAN) for high-resolution image inpainting. To enhance context reasoning, we construct the generator of AOT-GAN by stacking multiple layers of a proposed AOT block. For improving texture synthesis, we enhance the discriminator of AOT-GAN by training it with a tailored mask-prediction task.
  arXiv  Detail & Related papers  (2021-04-03T15:50:17Z)
• SieveNet: A Unified Framework for Robust Image-Based Virtual Try-On [14.198545992098309]
  SieveNet is a framework for robust image-based virtual try-on. We introduce a multi-stage coarse-to-fine warping network to better model fine-grained intricacies. We also introduce a try-on cloth conditioned segmentation mask prior to improve the texture transfer network.
  arXiv  Detail & Related papers  (2020-01-17T12:33:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.