SwiftAvatar: Efficient Auto-Creation of Parameterized Stylized Character
on Arbitrary Avatar Engines
- URL: http://arxiv.org/abs/2301.08153v1
- Date: Thu, 19 Jan 2023 16:14:28 GMT
- Title: SwiftAvatar: Efficient Auto-Creation of Parameterized Stylized Character
on Arbitrary Avatar Engines
- Authors: Shizun Wang, Weihong Zeng, Xu Wang, Hao Yang, Li Chen, Chuang Zhang,
Ming Wu, Yi Yuan, Yunzhao Zeng, Min Zheng
- Abstract summary: We propose SwiftAvatar, a novel avatar auto-creation framework.
We synthesize data in high-quality as many as possible, consisting of avatar vectors and their corresponding realistic faces.
Our experiments demonstrate the effectiveness and efficiency of SwiftAvatar on two different avatar engines.
- Score: 34.645129752596915
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The creation of a parameterized stylized character involves careful selection
of numerous parameters, also known as the "avatar vectors" that can be
interpreted by the avatar engine. Existing unsupervised avatar vector
estimation methods that auto-create avatars for users, however, often fail to
work because of the domain gap between realistic faces and stylized avatar
images. To this end, we propose SwiftAvatar, a novel avatar auto-creation
framework that is evidently superior to previous works. SwiftAvatar introduces
dual-domain generators to create pairs of realistic faces and avatar images
using shared latent codes. The latent codes can then be bridged with the avatar
vectors as pairs, by performing GAN inversion on the avatar images rendered
from the engine using avatar vectors. Through this way, we are able to
synthesize paired data in high-quality as many as possible, consisting of
avatar vectors and their corresponding realistic faces. We also propose
semantic augmentation to improve the diversity of synthesis. Finally, a
light-weight avatar vector estimator is trained on the synthetic pairs to
implement efficient auto-creation. Our experiments demonstrate the
effectiveness and efficiency of SwiftAvatar on two different avatar engines.
The superiority and advantageous flexibility of SwiftAvatar are also verified
in both subjective and objective evaluations.
Related papers
- A multidimensional measurement of photorealistic avatar quality of experience [14.94879852506943]
Photorealistic avatars are human avatars that look, move, and talk like real people.
We provide an open source test framework to subjectively measure photorealistic avatar performance in ten dimensions.
We show that the correlation of nine of these subjective metrics with PSNR, SSIM, LPIPS, FID, and FVD is weak, and moderate for emotion accuracy.
arXiv Detail & Related papers (2024-11-13T22:47:24Z) - AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text [71.09533176800707]
AvatarStudio is a coarse-to-fine generative model that generates explicit textured 3D meshes for animatable human avatars.
By effectively leveraging the synergy between the articulated mesh representation and the DensePose-conditional diffusion model, AvatarStudio can create high-quality avatars.
arXiv Detail & Related papers (2023-11-29T18:59:32Z) - MagicAvatar: Multimodal Avatar Generation and Animation [70.55750617502696]
MagicAvatar is a framework for multimodal video generation and animation of human avatars.
It disentangles avatar video generation into two stages: multimodal-to-motion and motion-to-video generation.
We demonstrate the flexibility of MagicAvatar through various applications, including text-guided and video-guided avatar generation.
arXiv Detail & Related papers (2023-08-28T17:56:18Z) - AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars
Using 2D Diffusion [34.609403685504944]
We present AvatarFusion, a framework for zero-shot text-to-avatar generation.
We use a latent diffusion model to provide pixel-level guidance for generating human-realistic avatars.
We also introduce a novel optimization method, called Pixel-Semantics Difference-Sampling (PS-DS), which semantically separates the generation of body and clothes.
arXiv Detail & Related papers (2023-07-13T02:19:56Z) - AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation [14.062402203105712]
AvatarBooth is a novel method for generating high-quality 3D avatars using text prompts or specific images.
Our key contribution is the precise avatar generation control by using dual fine-tuned diffusion models.
We present a multi-resolution rendering strategy that facilitates coarse-to-fine supervision of 3D avatar generation.
arXiv Detail & Related papers (2023-06-16T14:18:51Z) - DreamWaltz: Make a Scene with Complex 3D Animatable Avatars [68.49935994384047]
We present DreamWaltz, a novel framework for generating and animating complex 3D avatars given text guidance and parametric human body prior.
For animation, our method learns an animatable 3D avatar representation from abundant image priors of diffusion model conditioned on various poses.
arXiv Detail & Related papers (2023-05-21T17:59:39Z) - OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane
Rendering [81.55960827071661]
Controllability, generalizability and efficiency are the major objectives of constructing face avatars represented by neural implicit field.
We propose One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a generalized controllable tri-plane rendering solution.
arXiv Detail & Related papers (2023-03-26T09:12:03Z) - AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging [12.535634029277212]
We propose a novel self-supervised learning framework to create high-quality stylized 3D avatars.
Our results achieve much higher preference scores than previous work and close to those of manual creation.
arXiv Detail & Related papers (2022-11-15T00:43:45Z) - StylePeople: A Generative Model of Fullbody Human Avatars [59.42166744151461]
We propose a new type of full-body human avatars, which combines parametric mesh-based body model with a neural texture.
We show that such avatars can successfully model clothing and hair, which usually poses a problem for mesh-based approaches.
We then propose a generative model for such avatars that can be trained from datasets of images and videos of people.
arXiv Detail & Related papers (2021-04-16T20:43:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.