Retrieval-Augmented Gaussian Avatars: Improving Expression Generalization

• URL: http://arxiv.org/abs/2603.08645v1
• Date: Mon, 09 Mar 2026 17:24:11 GMT
• Title: Retrieval-Augmented Gaussian Avatars: Improving Expression Generalization
• Abstract summary: We introduce RAF (Retrieval-Augmented Faces), a simple training-time augmentation designed for template-free head avatars that learn deformation from data.<n>RAF replaces a subset of the subject's expression features with nearest-neighbor expressions retrieved from this bank while still reconstructing the subject's original frames.<n>RAF consistently improves expression fidelity over the baseline, in both self-driving and cross-driving scenarios.
• Score: 39.32685427601982
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Template-free animatable head avatars can achieve high visual fidelity by learning expression-dependent facial deformation directly from a subject's capture, avoiding parametric face templates and hand-designed blendshape spaces. However, since learned deformation is supervised only by the expressions observed for a single identity, these models suffer from limited expression coverage and often struggle when driven by motions that deviate from the training distribution. We introduce RAF (Retrieval-Augmented Faces), a simple training-time augmentation designed for template-free head avatars that learn deformation from data. RAF constructs a large unlabeled expression bank and, during training, replaces a subset of the subject's expression features with nearest-neighbor expressions retrieved from this bank while still reconstructing the subject's original frames. This exposes the deformation field to a broader range of expression conditions, encouraging stronger identity-expression decoupling and improving robustness to expression distribution shift without requiring paired cross-identity data, additional annotations, or architectural changes. We further analyze how retrieval augmentation increases expression diversity and validate retrieval quality with a user study showing that retrieved neighbors are perceptually closer in expression and pose. Experiments on the NeRSemble benchmark demonstrate that RAF consistently improves expression fidelity over the baseline, in both self-driving and cross-driving scenarios.

Related papers

• RegHead: Non-Humanoid Head Blendshapes via Feed-Forward Registration [64.34831170405567]
  We present RegHead, a framework for constructing semantic blendshape sets for animatable non-humanoid head avatars.<n>Our approach produces higher-fidelity expression meshes than baselines, while running orders of magnitude faster than optimization.
  arXiv  Detail & Related papers  (2026-07-13T23:16:29Z)
• PixelSmile: Toward Fine-Grained Facial Expression Editing [65.87252658984724]
  PixelSmile is a diffusion framework that disentangles expression semantics via fully symmetric joint training.<n>Experiments demonstrate that PixelSmile achieves superior disentanglement and robust identity preservation.
  arXiv  Detail & Related papers  (2026-03-26T17:59:04Z)
• ID-Consistent, Precise Expression Generation with Blendshape-Guided Diffusion [40.50436862878818]
  We present a diffusion-based framework that faithfully reimagines any subject under any particular facial expression.<n>Our adapter generalizes beyond basic emotions to subtle micro-expressions and expressive transitions, overlooked by prior works.<n>In addition, a pluggable Reference Adapter enables expression editing in real images by transferring the appearance from a reference frame during synthesis.
  arXiv  Detail & Related papers  (2025-10-06T11:20:56Z)
• SEREP: Semantic Facial Expression Representation for Robust In-the-Wild Capture and Retargeting [4.083283519300837]
  We propose SEREP, a model that disentangles expression from identity at the semantic level.<n>We train a model to predict expression from monocular images relying on a novel semi-supervised scheme using low quality synthetic data.<n>Our experiments show that SEREP outperforms state-of-the-art methods, capturing challenging expressions and transferring them to new identities.
  arXiv  Detail & Related papers  (2024-12-18T22:12:28Z)
• EmojiDiff: Advanced Facial Expression Control with High Identity Preservation in Portrait Generation [8.314556078632412]
  We introduce EmojiDiff, the first end-to-end solution that enables simultaneous control of extremely detailed expression (RGB-level) and high-fidelity identity in portrait generation.<n>For decoupled training, we innovate ID-irrelevant Data Iteration (IDI) to synthesize cross-identity expression pairs.<n>We also present ID-enhanced Contrast Alignment (ICA) for further fine-tuning.
  arXiv  Detail & Related papers  (2024-12-02T08:24:11Z)
• GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images [55.431697263581626]
  We introduce a novel Geometry-aware Facial Expression Translation framework, which is based on parametric 3D facial representations and can stably decoupled expression. We achieve higher-quality and more accurate facial expression transfer results compared to state-of-the-art methods, and demonstrate applicability of various poses and complex textures.
  arXiv  Detail & Related papers  (2023-08-07T09:03:35Z)
• LatentAvatar: Learning Latent Expression Code for Expressive Neural Head Avatar [60.363572621347565]
  We present LatentAvatar, an expressive neural head avatar driven by latent expression codes. LatentAvatar is able to capture challenging expressions and the subtle movement of teeth and even eyeballs.
  arXiv  Detail & Related papers  (2023-05-02T03:49:12Z)
• One-Shot High-Fidelity Talking-Head Synthesis with Deformable Neural Radiance Field [81.07651217942679]
  Talking head generation aims to generate faces that maintain the identity information of the source image and imitate the motion of the driving image. We propose HiDe-NeRF, which achieves high-fidelity and free-view talking-head synthesis.
  arXiv  Detail & Related papers  (2023-04-11T09:47:35Z)
• MorphGAN: One-Shot Face Synthesis GAN for Detecting Recognition Bias [13.162012586770576]
  We describe a simulator that applies specific head pose and facial expression adjustments to images of previously unseen people. We show that by augmenting small datasets of faces with new poses and expressions improves the recognition performance by up to 9% depending on the augmentation and data scarcity.
  arXiv  Detail & Related papers  (2020-12-09T18:43:03Z)
• LEED: Label-Free Expression Editing via Disentanglement [57.09545215087179]
  LEED framework is capable of editing the expression of both frontal and profile facial images without requiring any expression label. Two novel losses are designed for optimal expression disentanglement and consistent synthesis.
  arXiv  Detail & Related papers  (2020-07-17T13:36:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.