AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

• URL: http://arxiv.org/abs/2608.14130v1
• Date: Fri, 14 Aug 2026 09:36:28 GMT
• Title: AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations
• Abstract summary: We introduce the FACETS dataset and propose AlignFace, an interpretable, human-aligned, face similarity metric.<n>Experiments found AlignFace significantly improves alignment with human subpopulation perceptions compared to baseline metrics.
• Score: 13.228617286387887
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computer vision models for generated facial content, such as face editing and privacy protection, increasingly affect people, requiring similarity metrics that serve as faithful proxies for human perception. While perceptual evaluation has progressed from signal-based heuristics to representation-based metrics, current approaches are limited to behavioral modeling without cognitive alignment. They rely on implicit and spurious relations while assuming a universal observer, failing to account for inherent variations across diverse human populations. This leads to inaccurate evaluative models of stakeholders and misleading guidance for generative model debugging. Rather than treating perception as a black box, we leverage scientific findings from cognitive psychology of human face similarity perception: dependence on facial featural and configural attributes, nonlinear psychophysical response scaling, and own-group biases. We introduce the FACETS dataset and propose AlignFace, an interpretable, human-aligned, face similarity metric that encodes these cognitive principles through ante-hoc modeling. It employs visual-language modeling (VLM) to encode paired face images and text-based attributes, gated cross-attention (CA) to extract attribute-specific facial difference representations, concept bottleneck modeling (CBM) to constrain reasoning via interpretable face attributes, and neural generalized additive model (GAM) to model their nonlinear influence. Experiments found AlignFace significantly improves alignment with human subpopulation perceptions compared to baseline metrics, including recent domain-free learned perceptual metrics. By bridging learned representations and human cognitive processes, this work enables more transparent and aligned perceptual evaluation metrics for face images.

Related papers

• Human face perception reflects inverse-generative and naturalistic discriminative objectives [2.4126189670630733]
  Deep neural networks offer mechanistic hypotheses for human face perception.<n>We compared six neural network models sharing an architecture but trained on distinct tasks.<n>We tested model predictions against face-dissimilarity judgments from 864 human participants.
  arXiv  Detail & Related papers  (2026-05-12T18:06:57Z)
• Learning to Look before Learning to Like: Incorporating Human Visual Cognition into Aesthetic Quality Assessment [10.379213455076439]
  We introduce AestheticNet, a novel AQA paradigm that integrates human-like visual cognition and semantic perception with a two-pathway architecture.<n>The visual attention pathway, implemented as a gaze-aligned visual encoder (GAVE), pre-trained offline on eye-tracking data using resource-efficient contrast gaze alignment.<n>Visual attention provides a cognitive prior reflecting foreground/background structure, color cascade, brightness, and lighting.
  arXiv  Detail & Related papers  (2026-04-17T09:04:01Z)
• Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning [50.76723760768117]
  Existing human pose estimation methods cannot recover plausible close interactions from in-the-wild videos.<n>We find that human appearance can provide a straightforward cue to address these obstacles.<n>We propose a dual-branch optimization framework to reconstruct accurate interactive motions with plausible body contacts constrained by human appearances, social proxemics, and physical laws.
  arXiv  Detail & Related papers  (2025-07-03T12:19:26Z)
• Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs [65.93003087656754]
  VisFactor is a benchmark that digitizes 20 vision-centric subtests from a well-established cognitive psychology assessment.<n>We evaluate 20 frontier Multimodal Large Language Models (MLLMs) from GPT, Gemini, Claude, LLaMA, Qwen, and SEED families.<n>The best-performing model achieves a score of only 25.19 out of 100, with consistent failures on tasks such as mental rotation, spatial relation inference, and figure-ground discrimination.
  arXiv  Detail & Related papers  (2025-02-23T04:21:32Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• How Do You Perceive My Face? Recognizing Facial Expressions in Multi-Modal Context by Modeling Mental Representations [5.895694050664867]
  We introduce a novel approach for facial expression classification that goes beyond simple classification tasks. Our model accurately classifies a perceived face and synthesizes the corresponding mental representation perceived by a human when observing a face in context. We evaluate synthesized expressions in a human study, showing that our model effectively produces approximations of human mental representations.
  arXiv  Detail & Related papers  (2024-09-04T09:32:40Z)
• Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption [64.07607726562841]
  Existing multi-person human reconstruction approaches mainly focus on recovering accurate poses or avoiding penetration. In this work, we tackle the task of reconstructing closely interactive humans from a monocular video. We propose to leverage knowledge from proxemic behavior and physics to compensate the lack of visual information.
  arXiv  Detail & Related papers  (2024-04-17T11:55:45Z)
• UniAR: A Unified model for predicting human Attention and Responses on visual content [12.281060227170792]
  We propose UniAR -- a unified model of human attention and preference behavior across diverse visual content. We train UniAR on diverse public datasets spanning natural images, webpages, and graphic designs, and achieve SOTA performance on multiple benchmarks. Potential applications include providing instant feedback on the effectiveness of UIs/visual content, and enabling designers and content-creation models to optimize their creation for human-centric improvements.
  arXiv  Detail & Related papers  (2023-12-15T19:57:07Z)
• Subjective Face Transform using Human First Impressions [4.6852346957345015]
  This work uses generative models to find semantically meaningful edits to a face image that change perceived attributes.<n>We train on real and synthetic faces, evaluate for in-domain and out-of-domain images using predictive models and human ratings.
  arXiv  Detail & Related papers  (2023-09-27T03:21:07Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• A Linked Aggregate Code for Processing Faces (Revised Version) [0.0]
  A model of face representation is compared to experimental data on the perception of facial similarity. The results suggest that the LAC based similarity measure may offer a fertile starting point for further modelling studies of face representation in higher visual areas.
  arXiv  Detail & Related papers  (2020-09-17T13:29:25Z)
• Unsupervised Learning Facial Parameter Regressor for Action Unit Intensity Estimation via Differentiable Renderer [51.926868759681014]
  We present a framework to predict the facial parameters based on a bone-driven face model (BDFM) under different views. The proposed framework consists of a feature extractor, a generator, and a facial parameter regressor.
  arXiv  Detail & Related papers  (2020-08-20T09:49:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.