Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models

• URL: http://arxiv.org/abs/2609.00411v1
• Date: Mon, 31 Aug 2026 21:46:54 GMT
• Title: Unmasking Face Embeddings: Reading, Rendering and Naming with Foundation Models
• Abstract summary: Building on recent work on embedding compatibility across models, we use simple pre-computed linear transformations to connect FR models with off-the-shelf foundation models.<n>This interoperability exposes face embeddings as semantically and visually rich biometric representations, with implications for interpretability, retrieval, reconstruction, and template security.
• Score: 8.929662625846595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern face recognition (FR) owes much of its success to deep neural networks that learn to extract compact identity embeddings from face images. These models are typically trained for identity discrimination, producing embeddings that are highly effective for biometric matching but largely opaque to semantic interpretation. In contrast, foundation models, pretrained on broad visual or vision--language tasks, provide rich interfaces for describing, retrieving, generating, and organizing visual content. This contrast raises a natural question: what capabilities become available when face embeddings from domain-specific FR models are made interoperable with foundation models? Building on recent work on embedding compatibility across models, we use simple pre-computed linear transformations, estimated from paired embeddings alone, to connect existing FR models with off-the-shelf foundation models. Once aligned with a foundation model, a face embedding can be 'unmasked' in multiple ways, without training or modifying either model: it can be read in natural language, enabling free-form text queries over a gallery of FR embeddings; rendered into a face image that recovers a person's appearance, using an unmodified diffusion decoder; and converted to a name, enabling identification even in the absence of an enrolled face gallery. In effect, one linear transformation turns an identity embedding into a rich embedding for web-scale foundation models. This interoperability exposes face embeddings as semantically and visually rich biometric representations, with direct implications for interpretability, retrieval, reconstruction, and template security.

Related papers

• Enhancing Single-Image Facial Demorphing using Multimodal Large Language Models [8.088040801883015]
  Face recognition systems are increasingly vulnerable to morphing attacks, where a composite image is crafted to match multiple identities.<n>This paper presents a novel reference-free facial demorphing framework that leverages Multimodal Large Language Models.
  arXiv  Detail & Related papers  (2026-05-25T05:40:32Z)
• Are Face Embeddings Compatible Across Deep Neural Network Models? [8.929662625846595]
  We study whether different deep neural network (DNN) models can align face representations of one model with another.<n>Our findings reveal surprising cross-model compatibility.<n>These findings have implications for model interoperability, ensemble design, and biometric template security.
  arXiv  Detail & Related papers  (2026-04-08T16:44:42Z)
• CLIP-FTI: Fine-Grained Face Template Inversion via CLIP-Driven Attribute Conditioning [15.01028853162385]
  Face recognition systems store face templates for efficient matching.<n>Once leaked, these templates pose a threat: inverting them can yield surrogates that compromise privacy and enable impersonation.<n>We present CLIP-FTI, a CLIP-driven fine-grained attribute conditioning framework for face template inversion.
  arXiv  Detail & Related papers  (2025-12-17T13:26:34Z)
• UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition [42.86969216015855]
  Face recognition (FR) stands as one of the most crucial applications in computer vision.<n>We propose a framework to enhance intra-class diversity for synthetic face recognition, shortened as UIFace.<n> Experiments show that our method significantly surpasses previous approaches with even less training data and half the size of synthetic dataset.
  arXiv  Detail & Related papers  (2025-02-27T06:22:18Z)
• OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
  Diffusion models have demonstrated impressive performance in face restoration.<n>We propose OSDFace, a novel one-step diffusion model for face restoration.<n>Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2024-11-26T07:07:48Z)
• Arc2Face: A Foundation Model for ID-Consistent Human Faces [95.00331107591859]
  Arc2Face is an identity-conditioned face foundation model. It can generate diverse photo-realistic images with an unparalleled degree of face similarity than existing models.
  arXiv  Detail & Related papers  (2024-03-18T10:32:51Z)
• When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation [60.305112612629465]
  Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images. We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models. Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
  arXiv  Detail & Related papers  (2023-11-29T09:05:14Z)
• Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator [29.58245990622227]
  Multimodal-driven talking face generation refers to animating a portrait with the given pose, expression, and gaze transferred from the driving image and video, or estimated from the text and audio. Existing methods ignore the potential of text modal, and their generators mainly follow the source-oriented feature paradigm coupled with unstable GAN frameworks. We derive a novel paradigm free of unstable seesaw-style optimization, resulting in simple, stable, and effective training and inference schemes.
  arXiv  Detail & Related papers  (2023-05-04T07:01:36Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)
• InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs [73.27299786083424]
  We propose a framework called InterFaceGAN to interpret the disentangled face representation learned by state-of-the-art GAN models. We first find that GANs learn various semantics in some linear subspaces of the latent space. We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection.
  arXiv  Detail & Related papers  (2020-05-18T18:01:22Z)
• DotFAN: A Domain-transferred Face Augmentation Network for Pose and Illumination Invariant Face Recognition [94.96686189033869]
  We propose a 3D model-assisted domain-transferred face augmentation network (DotFAN) DotFAN can generate a series of variants of an input face based on the knowledge distilled from existing rich face datasets collected from other domains. Experiments show that DotFAN is beneficial for augmenting small face datasets to improve their within-class diversity.
  arXiv  Detail & Related papers  (2020-02-23T08:16:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.