HumanCoser: Layered 3D Human Generation via Semantic-Aware Diffusion Model

• URL: http://arxiv.org/abs/2408.11357v1
• Date: Wed, 21 Aug 2024 06:00:11 GMT
• Title: HumanCoser: Layered 3D Human Generation via Semantic-Aware Diffusion Model
• Authors: Yi Wang, Jian Ma, Ruizhi Shao, Qiao Feng, Yu-kun Lai, Kun Li,
• Abstract summary: This paper aims to generate physically-layered 3D humans from text prompts. We propose a novel layer-wise dressed human representation based on a physically-decoupled diffusion model. To match the clothing with different body shapes, we propose an SMPL-driven implicit field network.
• Score: 43.66218796152962
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper aims to generate physically-layered 3D humans from text prompts. Existing methods either generate 3D clothed humans as a whole or support only tight and simple clothing generation, which limits their applications to virtual try-on and part-level editing. To achieve physically-layered 3D human generation with reusable and complex clothing, we propose a novel layer-wise dressed human representation based on a physically-decoupled diffusion model. Specifically, to achieve layer-wise clothing generation, we propose a dual-representation decoupling framework for generating clothing decoupled from the human body, in conjunction with an innovative multi-layer fusion volume rendering method. To match the clothing with different body shapes, we propose an SMPL-driven implicit field deformation network that enables the free transfer and reuse of clothing. Extensive experiments demonstrate that our approach not only achieves state-of-the-art layered 3D human generation with complex clothing but also supports virtual try-on and layered human animation.

Related papers

• TELA: Text to Layer-wise 3D Clothed Human Generation [27.93447899876341]
  This paper addresses the task of 3D clothed human generation from textural descriptions. We propose a layer-wise clothed human representation combined with a progressive optimization strategy. Our approach achieves state-of-the-art 3D clothed human generation while also supporting cloth editing applications.
  arXiv  Detail & Related papers  (2024-04-25T17:05:38Z)
• Synthesizing Moving People with 3D Control [88.68284137105654]
  We present a diffusion model-based framework for animating people from a single image for a given target 3D motion sequence. For the first part, we learn an in-filling diffusion model to hallucinate unseen parts of a person given a single image. Second, we develop a diffusion-based rendering pipeline, which is controlled by 3D human poses.
  arXiv  Detail & Related papers  (2024-01-19T18:59:11Z)
• Layered 3D Human Generation via Semantic-Aware Diffusion Model [63.459666003261276]
  We propose a text-driven layered 3D human generation framework based on a novel semantic-aware diffusion model. To keep the generated clothing consistent with the target text, we propose a semantic-confidence strategy for clothing. To match the clothing with different body shapes, we propose a SMPL-driven implicit field deformation network.
  arXiv  Detail & Related papers  (2023-12-10T07:34:43Z)
• HumanLiff: Layer-wise 3D Human Generation with Diffusion Model [55.891036415316876]
  Existing 3D human generative models mainly generate a clothed 3D human as an undetectable 3D model in a single pass. We propose HumanLiff, the first layer-wise 3D human generative model with a unified diffusion process.
  arXiv  Detail & Related papers  (2023-08-18T17:59:04Z)
• Capturing and Animation of Body and Clothing from Monocular Video [105.87228128022804]
  We present SCARF, a hybrid model combining a mesh-based body with a neural radiance field. integrating the mesh into the rendering enables us to optimize SCARF directly from monocular videos. We demonstrate that SCARFs clothing with higher visual quality than existing methods, that the clothing deforms with changing body pose and body shape, and that clothing can be successfully transferred between avatars of different subjects.
  arXiv  Detail & Related papers  (2022-10-04T19:34:05Z)
• gDNA: Towards Generative Detailed Neural Avatars [94.9804106939663]
  We show that our model is able to generate natural human avatars wearing diverse and detailed clothing. Our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-01-11T18:46:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.