AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging

• URL: http://arxiv.org/abs/2211.07818v1
• Date: Tue, 15 Nov 2022 00:43:45 GMT
• Title: AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging
• Authors: Shen Sang, Tiancheng Zhi, Guoxian Song, Minghao Liu, Chunpong Lai, Jing Liu, Xiang Wen, James Davis, Linjie Luo
• Abstract summary: We propose a novel self-supervised learning framework to create high-quality stylized 3D avatars. Our results achieve much higher preference scores than previous work and close to those of manual creation.
• Score: 12.535634029277212
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stylized 3D avatars have become increasingly prominent in our modern life. Creating these avatars manually usually involves laborious selection and adjustment of continuous and discrete parameters and is time-consuming for average users. Self-supervised approaches to automatically create 3D avatars from user selfies promise high quality with little annotation cost but fall short in application to stylized avatars due to a large style domain gap. We propose a novel self-supervised learning framework to create high-quality stylized 3D avatars with a mix of continuous and discrete parameters. Our cascaded domain bridging framework first leverages a modified portrait stylization approach to translate input selfies into stylized avatar renderings as the targets for desired 3D avatars. Next, we find the best parameters of the avatars to match the stylized avatar renderings through a differentiable imitator we train to mimic the avatar graphics engine. To ensure we can effectively optimize the discrete parameters, we adopt a cascaded relaxation-and-search pipeline. We use a human preference study to evaluate how well our method preserves user identity compared to previous work as well as manual creation. Our results achieve much higher preference scores than previous work and close to those of manual creation. We also provide an ablation study to justify the design choices in our pipeline.

Related papers

• PuzzleAvatar: Assembling 3D Avatars from Personal Albums [54.831084076478874]
  We develop PuzzleAvatar, a novel model that generates a faithful 3D avatar from a personal OOTD album. We exploit the learned tokens as "puzzle pieces" from which we assemble a faithful, personalized 3D avatar.
  arXiv  Detail & Related papers  (2024-05-23T17:59:56Z)
• X-Oscar: A Progressive Framework for High-quality Text-guided 3D Animatable Avatar Generation [63.74194950823133]
  X-Oscar is a progressive framework for generating high-quality animatable avatars from text prompts. To tackle oversaturation, we introduce Adaptive Variational, representing avatars as an adaptive distribution during training. We also present Avatar-aware Score Distillation Sampling (ASDS), a novel technique that incorporates avatar-aware noise into rendered images.
  arXiv  Detail & Related papers  (2024-05-02T02:30:39Z)
• One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation [31.310769289315648]
  This paper introduces a novel approach to create high quality head avatar utilizing only a single or a few images per user. We learn a generative model for 3D animatable photo-realistic head avatar from a multi-view dataset of expressions from 2407 subjects. Our method demonstrates compelling results and outperforms existing state-of-the-art methods for few-shot avatar adaptation.
  arXiv  Detail & Related papers  (2024-02-19T07:48:29Z)
• AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text [71.09533176800707]
  AvatarStudio is a coarse-to-fine generative model that generates explicit textured 3D meshes for animatable human avatars. By effectively leveraging the synergy between the articulated mesh representation and the DensePose-conditional diffusion model, AvatarStudio can create high-quality avatars.
  arXiv  Detail & Related papers  (2023-11-29T18:59:32Z)
• AvatarVerse: High-quality & Stable 3D Avatar Creation from Text and Pose [23.76390935089982]
  We present AvatarVerse, a stable pipeline for generating high expressivequality 3D avatars from text descriptions and pose guidance. To this end, we propose zero-fidelity 3D modeling of 3D avatars that are not only more expressive, but also higher quality stablizes.
  arXiv  Detail & Related papers  (2023-08-07T14:09:46Z)
• AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation [14.062402203105712]
  AvatarBooth is a novel method for generating high-quality 3D avatars using text prompts or specific images. Our key contribution is the precise avatar generation control by using dual fine-tuned diffusion models. We present a multi-resolution rendering strategy that facilitates coarse-to-fine supervision of 3D avatar generation.
  arXiv  Detail & Related papers  (2023-06-16T14:18:51Z)
• DreamWaltz: Make a Scene with Complex 3D Animatable Avatars [68.49935994384047]
  We present DreamWaltz, a novel framework for generating and animating complex 3D avatars given text guidance and parametric human body prior. For animation, our method learns an animatable 3D avatar representation from abundant image priors of diffusion model conditioned on various poses.
  arXiv  Detail & Related papers  (2023-05-21T17:59:39Z)
• Text-Conditional Contextualized Avatars For Zero-Shot Personalization [47.85747039373798]
  We propose a pipeline that enables personalization of image generation with avatars capturing a user's identity in a delightful way. Our pipeline is zero-shot, avatar texture and style agnostic, and does not require training on the avatar at all. We show, for the first time, how to leverage large-scale image datasets to learn human 3D pose parameters.
  arXiv  Detail & Related papers  (2023-04-14T22:00:44Z)
• OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Rendering [81.55960827071661]
  Controllability, generalizability and efficiency are the major objectives of constructing face avatars represented by neural implicit field. We propose One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a generalized controllable tri-plane rendering solution.
  arXiv  Detail & Related papers  (2023-03-26T09:12:03Z)
• SwiftAvatar: Efficient Auto-Creation of Parameterized Stylized Character on Arbitrary Avatar Engines [34.645129752596915]
  We propose SwiftAvatar, a novel avatar auto-creation framework. We synthesize data in high-quality as many as possible, consisting of avatar vectors and their corresponding realistic faces. Our experiments demonstrate the effectiveness and efficiency of SwiftAvatar on two different avatar engines.
  arXiv  Detail & Related papers  (2023-01-19T16:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.