GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting

• URL: http://arxiv.org/abs/2405.07472v2
• Date: Thu, 23 May 2024 07:54:48 GMT
• Title: GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting
• Authors: Haodong Chen, Yongle Huang, Haojian Huang, Xiangsheng Ge, Dian Shao,
• Abstract summary: E-commerce has underscored the importance of Virtual Try-On (VTON) Research on 3D VTON primarily centers on garment-body shape compatibility. Thanks to advances in 3D scene editing, a 2D diffusion model has now been adapted for 3D editing via multi-viewpoint editing.
• Score: 2.2975420753582028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing prominence of e-commerce has underscored the importance of Virtual Try-On (VTON). However, previous studies predominantly focus on the 2D realm and rely heavily on extensive data for training. Research on 3D VTON primarily centers on garment-body shape compatibility, a topic extensively covered in 2D VTON. Thanks to advances in 3D scene editing, a 2D diffusion model has now been adapted for 3D editing via multi-viewpoint editing. In this work, we propose GaussianVTON, an innovative 3D VTON pipeline integrating Gaussian Splatting (GS) editing with 2D VTON. To facilitate a seamless transition from 2D to 3D VTON, we propose, for the first time, the use of only images as editing prompts for 3D editing. To further address issues, e.g., face blurring, garment inaccuracy, and degraded viewpoint quality during editing, we devise a three-stage refinement strategy to gradually mitigate potential issues. Furthermore, we introduce a new editing strategy termed Edit Recall Reconstruction (ERR) to tackle the limitations of previous editing strategies in leading to complex geometric changes. Our comprehensive experiments demonstrate the superiority of GaussianVTON, offering a novel perspective on 3D VTON while also establishing a novel starting point for image-prompting 3D scene editing.

Related papers

• DisCo3D: Distilling Multi-View Consistency for 3D Scene Editing [12.383291424229448]
  We propose textbfDisCo3D, a novel framework that distills 3D consistency priors into a 2D editor.<n>Our method first fine-tunes a 3D generator using multi-view inputs for scene adaptation, then trains a 2D editor through consistency distillation.<n> Experimental results show DisCo3D achieves stable multi-view consistency and outperforms state-of-the-art methods in editing quality.
  arXiv  Detail & Related papers  (2025-08-03T09:27:41Z)
• ARAP-GS: Drag-driven As-Rigid-As-Possible 3D Gaussian Splatting Editing with Diffusion Prior [7.218737495375119]
  ARAP-GS is a drag-driven 3DGS editing framework based on As-id-As-Rig (ARAP) deformation. We are the first to apply ARAP deformation directly to 3D Gaussians, enabling flexible, drag-driven geometric transformations. Our method is highly efficient, requiring only 10 to 20 minutes to edit a scene on a single 3090 GPU.
  arXiv  Detail & Related papers  (2025-04-17T09:37:11Z)
• GS-VTON: Controllable 3D Virtual Try-on with Gaussian Splatting [0.0]
  
  arXiv  Detail & Related papers  (2024-10-07T17:58:20Z)
• 3D Gaussian Editing with A Single Image [19.662680524312027]
  We introduce a novel single-image-driven 3D scene editing approach based on 3D Gaussian Splatting. Our method learns to optimize the 3D Gaussians to align with an edited version of the image rendered from a user-specified viewpoint. Experiments show the effectiveness of our method in handling geometric details, long-range, and non-rigid deformation.
  arXiv  Detail & Related papers  (2024-08-14T13:17:42Z)
• 3DEgo: 3D Editing on the Go! [6.072473323242202]
  We introduce 3DEgo to address a novel problem of directly synthesizing 3D scenes from monocular videos guided by textual prompts. Our framework streamlines the conventional multi-stage 3D editing process into a single-stage workflow. 3DEgo demonstrates remarkable editing precision, speed, and adaptability across a variety of video sources.
  arXiv  Detail & Related papers  (2024-07-14T07:03:50Z)
• Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side Images [72.70883914827687]
  Tailor3D is a novel pipeline that creates customized 3D assets from editable dual-side images. It provides a user-friendly, efficient solution for editing 3D assets, with each editing step taking only seconds to complete.
  arXiv  Detail & Related papers  (2024-07-08T17:59:55Z)
• DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representation [57.406031264184584]
  DragGaussian is a 3D object drag-editing framework based on 3D Gaussian Splatting. Our contributions include the introduction of a new task, the development of DragGaussian for interactive point-based 3D editing, and comprehensive validation of its effectiveness through qualitative and quantitative experiments.
  arXiv  Detail & Related papers  (2024-05-09T14:34:05Z)
• DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing [72.54566271694654]
  We consider the problem of editing 3D objects and scenes based on open-ended language instructions. A common approach to this problem is to use a 2D image generator or editor to guide the 3D editing process. This process is often inefficient due to the need for iterative updates of costly 3D representations.
  arXiv  Detail & Related papers  (2024-04-29T17:59:30Z)
• SHAP-EDITOR: Instruction-guided Latent 3D Editing in Seconds [73.91114735118298]
  Shap-Editor is a novel feed-forward 3D editing framework. We demonstrate that direct 3D editing in this space is possible and efficient by building a feed-forward editor network.
  arXiv  Detail & Related papers  (2023-12-14T18:59:06Z)
• Editing 3D Scenes via Text Prompts without Retraining [80.57814031701744]
  DN2N is a text-driven editing method that allows for the direct acquisition of a NeRF model with universal editing capabilities. Our method employs off-the-shelf text-based editing models of 2D images to modify the 3D scene images. Our method achieves multiple editing types, including but not limited to appearance editing, weather transition, material changing, and style transfer.
  arXiv  Detail & Related papers  (2023-09-10T02:31:50Z)
• 3DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models [71.25937799010407]
  We equip text-guided diffusion models to achieve 3D-consistent generation. We study 3D local editing and propose a two-step solution. We extend our model to perform one-shot novel view synthesis.
  arXiv  Detail & Related papers  (2022-11-25T13:50:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.