LAGA: Layered 3D Avatar Generation and Customization via Gaussian Splatting

• URL: http://arxiv.org/abs/2405.12663v1
• Date: Tue, 21 May 2024 10:24:06 GMT
• Title: LAGA: Layered 3D Avatar Generation and Customization via Gaussian Splatting
• Authors: Jia Gong, Shenyu Ji, Lin Geng Foo, Kang Chen, Hossein Rahmani, Jun Liu,
• Abstract summary: LAyered Gaussian Avatar (LAGA) is a framework enabling the creation of high-fidelity decomposable avatars with diverse garments. By decoupling garments from avatar, our framework empowers users to conviniently edit avatars at the garment level. Our approach surpasses existing methods in the generation of 3D clothed humans.
• Score: 18.613001290226773
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creating and customizing a 3D clothed avatar from textual descriptions is a critical and challenging task. Traditional methods often treat the human body and clothing as inseparable, limiting users' ability to freely mix and match garments. In response to this limitation, we present LAyered Gaussian Avatar (LAGA), a carefully designed framework enabling the creation of high-fidelity decomposable avatars with diverse garments. By decoupling garments from avatar, our framework empowers users to conviniently edit avatars at the garment level. Our approach begins by modeling the avatar using a set of Gaussian points organized in a layered structure, where each layer corresponds to a specific garment or the human body itself. To generate high-quality garments for each layer, we introduce a coarse-to-fine strategy for diverse garment generation and a novel dual-SDS loss function to maintain coherence between the generated garments and avatar components, including the human body and other garments. Moreover, we introduce three regularization losses to guide the movement of Gaussians for garment transfer, allowing garments to be freely transferred to various avatars. Extensive experimentation demonstrates that our approach surpasses existing methods in the generation of 3D clothed humans.

Related papers

• DAGSM: Disentangled Avatar Generation with GS-enhanced Mesh [102.84518904896737]
  DAGSM is a novel pipeline that generates disentangled human bodies and garments from the given text prompts. We first create the unclothed body, followed by a sequence of individual cloth generation based on the body. Experiments have demonstrated that DAGSM generates high-quality disentangled avatars, supports clothing replacement and realistic animation, and outperforms the baselines in visual quality.
  arXiv  Detail & Related papers  (2024-11-20T07:00:48Z)
• PuzzleAvatar: Assembling 3D Avatars from Personal Albums [54.831084076478874]
  We develop PuzzleAvatar, a novel model that generates a faithful 3D avatar from a personal OOTD album. We exploit the learned tokens as "puzzle pieces" from which we assemble a faithful, personalized 3D avatar.
  arXiv  Detail & Related papers  (2024-05-23T17:59:56Z)
• LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer [40.372917698238204]
  We present Layered Gaussian Avatars (LayGA), a new representation that formulates body and clothing as two separate layers. Our representation is built upon the Gaussian map-based avatar for its excellent representation power of garment details. In the single-layer reconstruction stage, we propose a series of geometric constraints to reconstruct smooth surfaces. In the multi-layer fitting stage, we train two separate models to represent body and clothing and utilize the reconstructed clothing geometries as 3D supervision.
  arXiv  Detail & Related papers  (2024-05-12T16:11:28Z)
• DivAvatar: Diverse 3D Avatar Generation with a Single Prompt [95.9978722953278]
  DivAvatar is a framework that generates diverse avatars from a single text prompt. It has two key designs that help achieve generation diversity and visual quality. Extensive experiments show that DivAvatar is highly versatile in generating avatars of diverse appearances.
  arXiv  Detail & Related papers  (2024-02-27T08:10:31Z)
• AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning [61.59722900152847]
  We introduce an approach for 3D head avatar generation and editing based on a 3D Generative Adversarial Network (GAN) and a Latent Diffusion Model (LDM) We exploit the conditioning capabilities of LDMs to enable multi-modal control over the latent space of a pre-trained 3D GAN. Our method can generate and edit 3D head avatars given a mixture of control signals such as RGB input, segmentation masks, and global attributes.
  arXiv  Detail & Related papers  (2024-02-08T16:41:20Z)
• AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusion [34.609403685504944]
  We present AvatarFusion, a framework for zero-shot text-to-avatar generation. We use a latent diffusion model to provide pixel-level guidance for generating human-realistic avatars. We also introduce a novel optimization method, called Pixel-Semantics Difference-Sampling (PS-DS), which semantically separates the generation of body and clothes.
  arXiv  Detail & Related papers  (2023-07-13T02:19:56Z)
• AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation [14.062402203105712]
  AvatarBooth is a novel method for generating high-quality 3D avatars using text prompts or specific images. Our key contribution is the precise avatar generation control by using dual fine-tuned diffusion models. We present a multi-resolution rendering strategy that facilitates coarse-to-fine supervision of 3D avatar generation.
  arXiv  Detail & Related papers  (2023-06-16T14:18:51Z)
• Capturing and Animation of Body and Clothing from Monocular Video [105.87228128022804]
  We present SCARF, a hybrid model combining a mesh-based body with a neural radiance field. integrating the mesh into the rendering enables us to optimize SCARF directly from monocular videos. We demonstrate that SCARFs clothing with higher visual quality than existing methods, that the clothing deforms with changing body pose and body shape, and that clothing can be successfully transferred between avatars of different subjects.
  arXiv  Detail & Related papers  (2022-10-04T19:34:05Z)
• gDNA: Towards Generative Detailed Neural Avatars [94.9804106939663]
  We show that our model is able to generate natural human avatars wearing diverse and detailed clothing. Our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-01-11T18:46:38Z)
• Explicit Clothing Modeling for an Animatable Full-Body Avatar [21.451440299450592]
  We build an animatable clothed body avatar with an explicit representation of the clothing on the upper body from multi-view captured videos. To learn the interaction between the body dynamics and clothing states, we use a temporal convolution network to predict the clothing latent code. We show photorealistic animation output for three different actors, and demonstrate the advantage of our clothed-body avatars over single-layer avatars.
  arXiv  Detail & Related papers  (2021-06-28T17:58:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.