VerLM: Explaining Face Verification Using Natural Language

• URL: http://arxiv.org/abs/2601.01798v1
• Date: Mon, 05 Jan 2026 05:16:07 GMT
• Title: VerLM: Explaining Face Verification Using Natural Language
• Authors: Syed Abdul Hannan, Hazim Bukhari, Thomas Cantalapiedra, Eman Ansar, Massa Baali, Rita Singh, Bhiksha Raj,
• Abstract summary: We introduce an innovative Vision-Language Model (VLM) for Face Verification.<n>Our model is uniquely trained using two complementary explanation styles.<n>The proposed VLM integrates sophisticated feature extraction techniques with advanced reasoning capabilities, enabling clear articulation of its verification process.
• Score: 50.56081916981731
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Face verification systems have seen substantial advancements; however, they often lack transparency in their decision-making processes. In this paper, we introduce an innovative Vision-Language Model (VLM) for Face Verification, which not only accurately determines if two face images depict the same individual but also explicitly explains the rationale behind its decisions. Our model is uniquely trained using two complementary explanation styles: (1) concise explanations that summarize the key factors influencing its decision, and (2) comprehensive explanations detailing the specific differences observed between the images. We adapt and enhance a state-of-the-art modeling approach originally designed for audio-based differentiation to suit visual inputs effectively. This cross-modal transfer significantly improves our model's accuracy and interpretability. The proposed VLM integrates sophisticated feature extraction techniques with advanced reasoning capabilities, enabling clear articulation of its verification process. Our approach demonstrates superior performance, surpassing baseline methods and existing models. These findings highlight the immense potential of vision language models in face verification set up, contributing to more transparent, reliable, and explainable face verification systems.

Related papers

• Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities? [61.533560295383786]
  Unified Multimodal Large Language Models (U-MLLMs) integrate understanding and generation within a single architecture.<n>We observe that U-MLLMs fail to maintain semantic equivalence when required to render the same results in the image modality.<n>We introduce VGUBench, a framework to decouple reasoning logic from generation fidelity.
  arXiv  Detail & Related papers  (2026-02-27T06:23:56Z)
• Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow [9.561772135477883]
  LVLMs (Large Vision-Language Models) follow the same architecture, with visual information gradually integrated into semantic representations.<n>We observe that the model's attention distribution does not exhibit sufficient emphasis on semantic representations.<n>This misalignment undermines the model's visual understanding ability and contributes to hallucinations.
  arXiv  Detail & Related papers  (2025-05-20T12:10:13Z)
• Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data [35.229595049396245]
  We propose a novel visual rejection sampling framework to improve the cognition and explainability of LMMs.<n>Our approach begins by synthesizing interpretable answers that include human-verifiable visual features.<n>After each round of fine-tuning, we apply a reward model-free filtering mechanism to select the highest-quality interpretable answers.
  arXiv  Detail & Related papers  (2025-02-19T19:05:45Z)
• From Pixels to Words: Leveraging Explainability in Face Recognition through Interactive Natural Language Processing [2.7568948557193287]
  Face Recognition (FR) has advanced significantly with the development of deep learning, achieving high accuracy in several applications.<n>The lack of interpretability of these systems raises concerns about their accountability, fairness, and reliability.<n>We propose an interactive framework to enhance the explainability of FR models by combining model-agnostic Explainable Artificial Intelligence (XAI) and Natural Language Processing (NLP) techniques.
  arXiv  Detail & Related papers  (2024-09-24T13:40:39Z)
• Diffexplainer: Towards Cross-modal Global Explanations with Diffusion Models [51.21351775178525]
  DiffExplainer is a novel framework that, leveraging language-vision models, enables multimodal global explainability. It employs diffusion models conditioned on optimized text prompts, synthesizing images that maximize class outputs. The analysis of generated visual descriptions allows for automatic identification of biases and spurious features.
  arXiv  Detail & Related papers  (2024-04-03T10:11:22Z)
• Bridging Generative and Discriminative Models for Unified Visual Perception with Diffusion Priors [56.82596340418697]
  We propose a simple yet effective framework comprising a pre-trained Stable Diffusion (SD) model containing rich generative priors, a unified head (U-head) capable of integrating hierarchical representations, and an adapted expert providing discriminative priors. Comprehensive investigations unveil potential characteristics of Vermouth, such as varying granularity of perception concealed in latent variables at distinct time steps and various U-net stages. The promising results demonstrate the potential of diffusion models as formidable learners, establishing their significance in furnishing informative and robust visual representations.
  arXiv  Detail & Related papers  (2024-01-29T10:36:57Z)
• Harnessing Diffusion Models for Visual Perception with Meta Prompts [68.78938846041767]
  We propose a simple yet effective scheme to harness a diffusion model for visual perception tasks. We introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception. Our approach achieves new performance records in depth estimation tasks on NYU depth V2 and KITTI, and in semantic segmentation task on CityScapes.
  arXiv  Detail & Related papers  (2023-12-22T14:40:55Z)
• Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problem [60.0878532426877]
  We propose a novel collaborative learning scheme from the viewpoint of visual perturbation calibration. Specifically, we devise a visual controller to construct two sorts of curated images with different perturbation extents. The experimental results on two diagnostic VQA-CP benchmark datasets evidently demonstrate its effectiveness.
  arXiv  Detail & Related papers  (2022-07-24T23:50:52Z)
• Deep Collaborative Multi-Modal Learning for Unsupervised Kinship Estimation [53.62256887837659]
  Kinship verification is a long-standing research challenge in computer vision. We propose a novel deep collaborative multi-modal learning (DCML) to integrate the underlying information presented in facial properties. Our DCML method is always superior to some state-of-the-art kinship verification methods.
  arXiv  Detail & Related papers  (2021-09-07T01:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.