GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details

• URL: http://arxiv.org/abs/2411.03047v1
• Date: Tue, 05 Nov 2024 12:30:07 GMT
• Title: GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details
• Authors: Zhongjin Luo, Haolin Liu, Chenghong Li, Wanghao Du, Zirong Jin, Wanhu Sun, Yinyu Nie, Weikai Chen, Xiaoguang Han,
• Abstract summary: GarVerseLOD aims to achieve unprecedented robustness in high-fidelity 3D garment reconstruction from a single unconstrained image. GarVerseLOD collects 6,000 high-quality cloth models with fine-grained geometry details manually created by professional artists. We propose a novel labeling paradigm based on conditional diffusion models to generate extensive paired images for each garment model with high photorealism.
• Score: 21.959372614365908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural implicit functions have brought impressive advances to the state-of-the-art of clothed human digitization from multiple or even single images. However, despite the progress, current arts still have difficulty generalizing to unseen images with complex cloth deformation and body poses. In this work, we present GarVerseLOD, a new dataset and framework that paves the way to achieving unprecedented robustness in high-fidelity 3D garment reconstruction from a single unconstrained image. Inspired by the recent success of large generative models, we believe that one key to addressing the generalization challenge lies in the quantity and quality of 3D garment data. Towards this end, GarVerseLOD collects 6,000 high-quality cloth models with fine-grained geometry details manually created by professional artists. In addition to the scale of training data, we observe that having disentangled granularities of geometry can play an important role in boosting the generalization capability and inference accuracy of the learned model. We hence craft GarVerseLOD as a hierarchical dataset with levels of details (LOD), spanning from detail-free stylized shape to pose-blended garment with pixel-aligned details. This allows us to make this highly under-constrained problem tractable by factorizing the inference into easier tasks, each narrowed down with smaller searching space. To ensure GarVerseLOD can generalize well to in-the-wild images, we propose a novel labeling paradigm based on conditional diffusion models to generate extensive paired images for each garment model with high photorealism. We evaluate our method on a massive amount of in-the-wild images. Experimental results demonstrate that GarVerseLOD can generate standalone garment pieces with significantly better quality than prior approaches. Project page: https://garverselod.github.io/

Related papers

• NGL-Prompter: Training-Free Sewing Pattern Estimation from a Single Image [4.620470560214746]
  Estimating sewing patterns from images is a practical approach for creating high-quality 3D garments.<n>We propose NGL (Natural Garment Language), a novel intermediate language that restructures GarmentCode into a representation more understandable to language models.<n>We evaluate our method on the Dress4D, CloSe and a newly collected dataset of approximately 5,000 in-the-wild fashion images.
  arXiv  Detail & Related papers  (2026-02-24T09:01:11Z)
• DexGarmentLab: Dexterous Garment Manipulation Environment with Generalizable Policy [74.9519138296936]
  Garment manipulation is a critical challenge due to the diversity in garment categories, geometries, and deformations.<n>We propose DexGarmentLab, the first environment specifically designed for dexterous (especially bimanual) garment manipulation.<n>It features large-scale high-quality 3D assets for 15 task scenarios, and refines simulation techniques tailored for garment modeling to reduce the sim-to-real gap.
  arXiv  Detail & Related papers  (2025-05-16T09:26:59Z)
• GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing [85.67881477813592]
  GarmentCrafter is a new approach that enables non-professional users to create and modify 3D garments from a single-view image. Our method achieves superior visual fidelity and inter-view coherence compared to state-of-the-art single-view 3D garment reconstruction methods.
  arXiv  Detail & Related papers  (2025-03-11T17:56:03Z)
• Improving Virtual Try-On with Garment-focused Diffusion Models [91.95830983115474]
  Diffusion models have led to the revolutionizing of generative modeling in numerous image synthesis tasks. We shape a new Diffusion model, namely GarDiff, which triggers the garment-focused diffusion process. Experiments on VITON-HD and DressCode datasets demonstrate the superiority of our GarDiff when compared to state-of-the-art VTON approaches.
  arXiv  Detail & Related papers  (2024-09-12T17:55:11Z)
• GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details [31.92583566128599]
  Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and time-consuming processes. We propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate 3D garment from text prompts.
  arXiv  Detail & Related papers  (2024-05-20T23:54:28Z)
• Diffusion Models are Efficient Data Generators for Human Mesh Recovery [55.37787289869703]
  We show that synthetic data created by generative models is complementary to CG-rendered data.<n>We propose an effective data generation pipeline based on recent diffusion models, termed HumanWild.<n>Our work could pave the way for scaling up 3D human recovery to in-the-wild scenes.
  arXiv  Detail & Related papers  (2024-03-17T06:31:16Z)
• A Generative Multi-Resolution Pyramid and Normal-Conditioning 3D Cloth Draping [37.77353302404437]
  We build a conditional variational autoencoder for 3D garment generation and draping. We propose a pyramid network to add garment details progressively in a canonical space. Our results on two public datasets, CLOTH3D and CAPE, show that our model is robust, controllable in terms of detail generation.
  arXiv  Detail & Related papers  (2023-11-05T16:12:48Z)
• Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single Images [19.43767376835559]
  A common problem for implicit-based methods is that they cannot produce separated and topology-consistent mesh for each garment piece. We proposed a novel geometry inference framework ReEF that reconstructs topology-consistent layered garment mesh by registering the explicit garment template to the whole-body implicit fields predicted from single images.
  arXiv  Detail & Related papers  (2022-03-28T18:13:01Z)
• Facial Geometric Detail Recovery via Implicit Representation [147.07961322377685]
  We present a robust texture-guided geometric detail recovery approach using only a single in-the-wild facial image. Our method combines high-quality texture completion with the powerful expressiveness of implicit surfaces. Our method not only recovers accurate facial details but also decomposes normals, albedos, and shading parts in a self-supervised way.
  arXiv  Detail & Related papers  (2022-03-18T01:42:59Z)
• gDNA: Towards Generative Detailed Neural Avatars [94.9804106939663]
  We show that our model is able to generate natural human avatars wearing diverse and detailed clothing. Our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-01-11T18:46:38Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• OSTeC: One-Shot Texture Completion [86.23018402732748]
  We propose an unsupervised approach for one-shot 3D facial texture completion. The proposed approach rotates an input image in 3D and fill-in the unseen regions by reconstructing the rotated image in a 2D face generator. We frontalize the target image by projecting the completed texture into the generator.
  arXiv  Detail & Related papers  (2020-12-30T23:53:26Z)
• Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images [50.34202789543989]
  Deep Fashion3D is the largest collection to date of 3D garment models. It provides rich annotations including 3D feature lines, 3D body pose and the corresponded multi-view real images. A novel adaptable template is proposed to enable the learning of all types of clothing in a single network.
  arXiv  Detail & Related papers  (2020-03-28T09:20:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.