LUCAS: Layered Universal Codec Avatars

• URL: http://arxiv.org/abs/2502.19739v2
• Date: Mon, 17 Mar 2025 20:38:27 GMT
• Title: LUCAS: Layered Universal Codec Avatars
• Authors: Di Liu, Teng Deng, Giljoo Nam, Yu Rong, Stanislav Pidhorskyi, Junxuan Li, Jason Saragih, Dimitris N. Metaxas, Chen Cao,
• Abstract summary: Photo 3D head avatar reconstruction faces critical challenges in modeling dynamic face-hair interactions.<n>We present LUCAS, a novel Universal Prior Model (UPM) for avatar modeling that disentangles face and hair through a layered representation.
• Score: 48.737625335705644
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photorealistic 3D head avatar reconstruction faces critical challenges in modeling dynamic face-hair interactions and achieving cross-identity generalization, particularly during expressions and head movements. We present LUCAS, a novel Universal Prior Model (UPM) for codec avatar modeling that disentangles face and hair through a layered representation. Unlike previous UPMs that treat hair as an integral part of the head, our approach separates the modeling of the hairless head and hair into distinct branches. LUCAS is the first to introduce a mesh-based UPM, facilitating real-time rendering on devices. Our layered representation also improves the anchor geometry for precise and visually appealing Gaussian renderings. Experimental results indicate that LUCAS outperforms existing single-mesh and Gaussian-based avatar models in both quantitative and qualitative assessments, including evaluations on held-out subjects in zero-shot driving scenarios. LUCAS demonstrates superior dynamic performance in managing head pose changes, expression transfer, and hairstyle variations, thereby advancing the state-of-the-art in 3D head avatar reconstruction.

Related papers

• Synthetic Prior for Few-Shot Drivable Head Avatar Inversion [61.51887011274453]
  We present SynShot, a novel method for the few-shot inversion of a drivable head avatar based on a synthetic prior.<n>Inspired by machine learning models trained solely on synthetic data, we propose a method that learns a prior model from a large dataset of synthetic heads.<n>We model the head avatar using 3D Gaussian splatting and a convolutional encoder-decoder that outputs Gaussian parameters in UV texture space.
  arXiv  Detail & Related papers  (2025-01-12T19:01:05Z)
• GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations [54.94362657501809]
  We propose a new method to generate highly dynamic and deformable human head avatars from multi-view imagery in real-time. At the core of our method is a hierarchical representation of head models that allows to capture the complex dynamics of facial expressions and head movements. We train this coarse-to-fine facial avatar model along with the head pose as a learnable parameter in an end-to-end framework.
  arXiv  Detail & Related papers  (2024-09-18T13:05:43Z)
• GPHM: Gaussian Parametric Head Model for Monocular Head Avatar Reconstruction [47.113910048252805]
  High-fidelity 3D human head avatars are crucial for applications in VR/AR, digital human, and film production. Recent advances have leveraged morphable face models to generate animated head avatars, representing varying identities and expressions. We introduce 3D Gaussian Parametric Head Model, which employs 3D Gaussians to accurately represent the complexities of the human head.
  arXiv  Detail & Related papers  (2024-07-21T06:03:11Z)
• MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing [34.31657241047574]
  We propose a Hybrid Mesh-Gaussian Head Avatar (MeGA) that models different head components with more suitable representations. MeGA generates higher-fidelity renderings for the whole head and naturally supports more downstream tasks. Experiments on the NeRSemble dataset demonstrate the effectiveness of our designs.
  arXiv  Detail & Related papers  (2024-04-29T18:10:12Z)
• HHAvatar: Gaussian Head Avatar with Dynamic Hairs [27.20228210350169]
  We proposeAvatar represented by controllable 3D Gaussians for high-fidelity head avatar with dynamic hair modeling. Our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution.
  arXiv  Detail & Related papers  (2023-12-05T11:01:44Z)
• DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars [48.50728107738148]
  DiffusionAvatars synthesizes a high-fidelity 3D head avatar of a person, offering intuitive control over both pose and expression. For coarse guidance of the expression and head pose, we render a neural parametric head model (NPHM) from the target viewpoint. We condition DiffusionAvatars directly on the expression codes obtained from NPHM via cross-attention.
  arXiv  Detail & Related papers  (2023-11-30T15:43:13Z)
• Generalizable One-shot Neural Head Avatar [90.50492165284724]
  We present a method that reconstructs and animates a 3D head avatar from a single-view portrait image. We propose a framework that not only generalizes to unseen identities based on a single-view image, but also captures characteristic details within and beyond the face area.
  arXiv  Detail & Related papers  (2023-06-14T22:33:09Z)
• One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Field [81.07651217942679]
  Talking head generation aims to generate faces that maintain the identity information of the source image and imitate the motion of the driving image. We propose HiDe-NeRF, which achieves high-fidelity and free-view talking-head synthesis.
  arXiv  Detail & Related papers  (2023-04-11T09:47:35Z)
• GANHead: Towards Generative Animatable Neural Head Avatars [31.35233032284164]
  GANHead is a novel generative head model that takes advantages of the fine-grained control over the explicit expression parameters. It represents coarse geometry, fine-gained details and texture via three networks in canonical space. It achieves superior performance on head avatar generation and raw scan fitting.
  arXiv  Detail & Related papers  (2023-04-08T07:56:21Z)
• Free-HeadGAN: Neural Talking Head Synthesis with Explicit Gaze Control [54.079327030892244]
  Free-HeadGAN is a person-generic neural talking head synthesis system. We show that modeling faces with sparse 3D facial landmarks are sufficient for achieving state-of-the-art generative performance.
  arXiv  Detail & Related papers  (2022-08-03T16:46:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.