PERSE: Personalized 3D Generative Avatars from A Single Portrait

• URL: http://arxiv.org/abs/2412.21206v1
• Date: Mon, 30 Dec 2024 18:59:58 GMT
• Title: PERSE: Personalized 3D Generative Avatars from A Single Portrait
• Authors: Hyunsoo Cha, Inhee Lee, Hanbyul Joo,
• Abstract summary: PERSE is a method for building an animatable personalized generative avatar from a reference portrait. Our method begins by synthesizing large-scale synthetic 2D video datasets. We propose a novel pipeline to produce high-quality, photorealistic 2D videos with facial attribute editing.
• Score: 7.890834685325639
• License:
• Abstract: We present PERSE, a method for building an animatable personalized generative avatar from a reference portrait. Our avatar model enables facial attribute editing in a continuous and disentangled latent space to control each facial attribute, while preserving the individual's identity. To achieve this, our method begins by synthesizing large-scale synthetic 2D video datasets, where each video contains consistent changes in the facial expression and viewpoint, combined with a variation in a specific facial attribute from the original input. We propose a novel pipeline to produce high-quality, photorealistic 2D videos with facial attribute editing. Leveraging this synthetic attribute dataset, we present a personalized avatar creation method based on the 3D Gaussian Splatting, learning a continuous and disentangled latent space for intuitive facial attribute manipulation. To enforce smooth transitions in this latent space, we introduce a latent space regularization technique by using interpolated 2D faces as supervision. Compared to previous approaches, we demonstrate that PERSE generates high-quality avatars with interpolated attributes while preserving identity of reference person.

Related papers

• Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance [69.9745497000557]
  We introduce Arc2Avatar, the first SDS-based method utilizing a human face foundation model as guidance with just a single image as input. Our avatars maintain a dense correspondence with a human face mesh template, allowing blendshape-based expression generation.
  arXiv  Detail & Related papers  (2025-01-09T17:04:33Z)
• FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model [45.0201701977516]
  Video-driven 3D facial animation transfer aims to drive avatars to reproduce the expressions of actors. We propose FreeAvatar, a robust facial animation transfer method that relies solely on our learned expression representation.
  arXiv  Detail & Related papers  (2024-09-20T03:17:01Z)
• DEGAS: Detailed Expressions on Full-Body Gaussian Avatars [13.683836322899953]
  We present DEGAS, the first 3D Gaussian Splatting (3DGS)-based modeling method for full-body avatars with rich facial expressions. We propose to adopt the expression latent space trained solely on 2D portrait images, bridging the gap between 2D talking faces and 3D avatars.
  arXiv  Detail & Related papers  (2024-08-20T06:52:03Z)
• PEGASUS: Personalized Generative 3D Avatars with Composable Attributes [9.493003322829718]
  We present a method for constructing a personalized generative 3D face avatar from monocular video sources. Our approach consists of two stages: synthetic database generation and constructing a personalized generative avatar.
  arXiv  Detail & Related papers  (2024-02-16T12:35:35Z)
• AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing [79.38471599977011]
  We propose AttriHuman-3D, an editable 3D human generation model. It generates all attributes in an overall attribute space with six feature planes, which are decomposed and manipulated with different attribute indexes. Our model provides a strong disentanglement between different attributes, allows fine-grained image editing and generates high-quality 3D human avatars.
  arXiv  Detail & Related papers  (2023-12-03T03:20:10Z)
• GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar [48.21353924040671]
  We propose to learn person-specific animatable avatars from images without assuming to have access to precise facial expression tracking. We learn a mapping from 3DMM facial expression parameters to the latent space of the generative model. With this scheme, we decouple 3D appearance reconstruction and animation control to achieve high fidelity in image synthesis.
  arXiv  Detail & Related papers  (2023-11-22T19:13:00Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• 3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image [142.047662926209]
  We generate 3D cartoon face shapes from single 2D GAN generated human faces. We manipulate latent codes to generate images with different poses and lighting, such that we can reconstruct the 3D cartoon face shapes.
  arXiv  Detail & Related papers  (2022-07-29T01:06:21Z)
• 3D GAN Inversion for Controllable Portrait Image Animation [45.55581298551192]
  We leverage newly developed 3D GANs, which allow explicit control over the pose of the image subject with multi-view consistency. The proposed technique for portrait image animation outperforms previous methods in terms of image quality, identity preservation, and pose transfer.
  arXiv  Detail & Related papers  (2022-03-25T04:06:06Z)
• Learning an Animatable Detailed 3D Face Model from In-The-Wild Images [50.09971525995828]
  We present the first approach to jointly learn a model with animatable detail and a detailed 3D face regressor from in-the-wild images. Our DECA model is trained to robustly produce a UV displacement map from a low-dimensional latent representation. We introduce a novel detail-consistency loss to disentangle person-specific details and expression-dependent wrinkles.
  arXiv  Detail & Related papers  (2020-12-07T19:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.