GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing

• URL: http://arxiv.org/abs/2503.08678v1
• Date: Tue, 11 Mar 2025 17:56:03 GMT
• Title: GarmentCrafter: Progressive Novel View Synthesis for Single-View 3D Garment Reconstruction and Editing
• Authors: Yuanhao Wang, Cheng Zhang, Gonçalo Frazão, Jinlong Yang, Alexandru-Eugen Ichim, Thabo Beeler, Fernando De la Torre,
• Abstract summary: GarmentCrafter is a new approach that enables non-professional users to create and modify 3D garments from a single-view image.<n>Our method achieves superior visual fidelity and inter-view coherence compared to state-of-the-art single-view 3D garment reconstruction methods.
• Score: 85.67881477813592
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce GarmentCrafter, a new approach that enables non-professional users to create and modify 3D garments from a single-view image. While recent advances in image generation have facilitated 2D garment design, creating and editing 3D garments remains challenging for non-professional users. Existing methods for single-view 3D reconstruction often rely on pre-trained generative models to synthesize novel views conditioning on the reference image and camera pose, yet they lack cross-view consistency, failing to capture the internal relationships across different views. In this paper, we tackle this challenge through progressive depth prediction and image warping to approximate novel views. Subsequently, we train a multi-view diffusion model to complete occluded and unknown clothing regions, informed by the evolving camera pose. By jointly inferring RGB and depth, GarmentCrafter enforces inter-view coherence and reconstructs precise geometries and fine details. Extensive experiments demonstrate that our method achieves superior visual fidelity and inter-view coherence compared to state-of-the-art single-view 3D garment reconstruction methods.

Related papers

• Single View Garment Reconstruction Using Diffusion Mapping Via Pattern Coordinates [45.48311596587306]
  Reconstructing 3D clothed humans from images is fundamental to applications like virtual try-on, avatar creation, and mixed reality. We present a novel method for high-fidelity 3D garment reconstruction from single images that bridges 2D and 3D representations.
  arXiv  Detail & Related papers  (2025-04-11T08:39:18Z)
• ExScene: Free-View 3D Scene Reconstruction with Gaussian Splatting from a Single Image [4.366356163044466]
  Existing methods are often limited to reconstruct low-consistency 3D scenes with narrow fields of view from single-view input. We propose ExScene, a two-stage pipeline to reconstruct an immersive 3D scene from any given single-view image. ExScene achieves consistent and immersive scene reconstruction using only single-view input.
  arXiv  Detail & Related papers  (2025-03-31T09:33:22Z)
• EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wild [79.71523320368388]
  Our work aims to reconstruct hand-object interactions from a single-view image. We first design a novel pipeline to estimate the underlying hand pose and object shape. With the initial reconstruction, we employ a prior-guided optimization scheme.
  arXiv  Detail & Related papers  (2024-11-21T16:33:35Z)
• ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis [63.169364481672915]
  We propose textbfViewCrafter, a novel method for synthesizing high-fidelity novel views of generic scenes from single or sparse images. Our method takes advantage of the powerful generation capabilities of video diffusion model and the coarse 3D clues offered by point-based representation to generate high-quality video frames.
  arXiv  Detail & Related papers  (2024-09-03T16:53:19Z)
• iFusion: Inverting Diffusion for Pose-Free Reconstruction from Sparse Views [61.707755434165335]
  iFusion is a novel 3D object reconstruction framework that requires only two views with unknown camera poses. We harness a pre-trained novel view synthesis diffusion model, which embeds implicit knowledge about the geometry and appearance of diverse objects. Experiments demonstrate strong performance in both pose estimation and novel view synthesis.
  arXiv  Detail & Related papers  (2023-12-28T18:59:57Z)
• Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairs [57.492124844326206]
  This work delves into the task of pose-free novel view synthesis from stereo pairs, a challenging and pioneering task in 3D vision. Our innovative framework, unlike any before, seamlessly integrates 2D correspondence matching, camera pose estimation, and NeRF rendering, fostering a synergistic enhancement of these tasks.
  arXiv  Detail & Related papers  (2023-12-12T13:22:44Z)
• High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
  We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views. Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
  arXiv  Detail & Related papers  (2022-11-28T18:59:52Z)
• Multi-person Implicit Reconstruction from a Single Image [37.6877421030774]
  We present a new end-to-end learning framework to obtain detailed and spatially coherent reconstructions of multiple people from a single image. Existing multi-person methods suffer from two main drawbacks: they are often model-based and cannot capture accurate 3D models of people with loose clothing and hair.
  arXiv  Detail & Related papers  (2021-04-19T13:21:55Z)
• Multi-View Consistency Loss for Improved Single-Image 3D Reconstruction of Clothed People [36.30755368202957]
  We present a novel method to improve the accuracy of the 3D reconstruction of clothed human shape from a single image. The accuracy and completeness for reconstruction of clothed people is limited due to the large variation in shape resulting from clothing, hair, body size, pose and camera viewpoint.
  arXiv  Detail & Related papers  (2020-09-29T17:18:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.