Benchmarking Multimodal Large Language Models for Face Recognition

• URL: http://arxiv.org/abs/2510.14866v1
• Date: Thu, 16 Oct 2025 16:42:27 GMT
• Title: Benchmarking Multimodal Large Language Models for Face Recognition
• Authors: Hatef Otroshi Shahreza, Sébastien Marcel,
• Abstract summary: Multimodal large language models (MLLMs) have achieved remarkable performance across diverse vision-and-language tasks.<n>We present a systematic benchmark of state-of-the-art MLLMs for face recognition on several face recognition datasets.
• Score: 44.02544110500887
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal large language models (MLLMs) have achieved remarkable performance across diverse vision-and-language tasks. However, their potential in face recognition remains underexplored. In particular, the performance of open-source MLLMs needs to be evaluated and compared with existing face recognition models on standard benchmarks with similar protocol. In this work, we present a systematic benchmark of state-of-the-art MLLMs for face recognition on several face recognition datasets, including LFW, CALFW, CPLFW, CFP, AgeDB and RFW. Experimental results reveal that while MLLMs capture rich semantic cues useful for face-related tasks, they lag behind specialized models in high-precision recognition scenarios in zero-shot applications. This benchmark provides a foundation for advancing MLLM-based face recognition, offering insights for the design of next-generation models with higher accuracy and generalization. The source code of our benchmark is publicly available in the project page.

Related papers

• Evaluating Multimodal Large Language Models for Heterogeneous Face Recognition [45.12459792999638]
  Multimodal Large Language Models (MLLMs) have recently demonstrated strong performance on a wide range of vision-language tasks.<n>We benchmark multiple open-source MLLMs across several cross-modality scenarios, including VIS-NIR, VIS-SWIR, and VIS-AL face recognition.<n>Our results reveal substantial performance gaps between MLLMs and classical face recognition systems, particularly under challenging cross-spectral conditions.
  arXiv  Detail & Related papers  (2026-01-21T19:17:21Z)
• FaceLLM: A Multimodal Large Language Model for Face Understanding [22.8742248559748]
  We introduce FaceLLM, a multimodal large language model trained specifically for facial image understanding.<n>To construct the training data, we propose a novel weakly supervised pipeline that uses ChatGPT with attribute-aware prompts to generate high-quality question-answer pairs.<n>Our experiments demonstrate that FaceLLM improves the performance of MLLMs on various face-centric tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-07-14T14:04:14Z)
• FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs [38.2031868024552]
  We introduce FaceBench, a dataset featuring hierarchical multi-view and multi-level attributes to assess the comprehensive face perception abilities of MLLMs.<n>Based on the structure, the proposed FaceBench consists of 49,919 visual question-answering (VQA) pairs for evaluation and 23,841 pairs for fine-tuning.<n>We further develop a robust face perception MLLM baseline, Face-LLaVA, by training with our proposed face VQA data.
  arXiv  Detail & Related papers  (2025-03-27T12:45:44Z)
• Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs [65.93003087656754]
  VisFactor is a benchmark that digitizes 20 vision-centric subtests from a well-established cognitive psychology assessment.<n>We evaluate 20 frontier Multimodal Large Language Models (MLLMs) from GPT, Gemini, Claude, LLaMA, Qwen, and SEED families.<n>The best-performing model achieves a score of only 25.19 out of 100, with consistent failures on tasks such as mental rotation, spatial relation inference, and figure-ground discrimination.
  arXiv  Detail & Related papers  (2025-02-23T04:21:32Z)
• Human Re-ID Meets LVLMs: What can we expect? [14.370360290704197]
  We compare the performance of the leading large vision-language models in the human re-identification task.<n>Our results confirm the strengths of LVLMs, but also their severe limitations that often lead to catastrophic answers.
  arXiv  Detail & Related papers  (2025-01-30T19:00:40Z)
• Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation [109.5893580175657]
  In recent times, the standard practice for developing MLLMs is to feed features from vision encoder(s) into the LLM and train with natural language supervision.<n>This approach often causes models to lean towards language comprehension and undermine the rich visual perception signals present in the data.<n>We propose VisPer-LM, the first approach that infuses visual perception knowledge from expert vision encoders into the LLM's hidden representations.
  arXiv  Detail & Related papers  (2024-12-12T18:55:18Z)
• Face-MLLM: A Large Face Perception Model [53.9441375205716]
  multimodal large language models (MLLMs) have achieved promising results on a wide range of vision-language tasks, but their ability to perceive and understand human faces is rarely explored. In this work, we comprehensively evaluate existing MLLMs on face perception tasks. Our model surpasses previous MLLMs on five famous face perception tasks.
  arXiv  Detail & Related papers  (2024-10-28T04:19:32Z)
• Tell Me Where You Are: Multimodal LLMs Meet Place Recognition [11.421492098416538]
  We introduce multimodal large language models (MLLMs) to visual place recognition (VPR) Our key design is to use vision-based retrieval to propose several candidates and then leverage language-based reasoning to carefully inspect each candidate for a final decision. Our results on three datasets demonstrate that integrating the general-purpose visual features from VFMs with the reasoning capabilities of MLLMs already provides an effective place recognition solution.
  arXiv  Detail & Related papers  (2024-06-25T12:59:46Z)
• RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition [78.97487780589574]
  Multimodal Large Language Models (MLLMs) excel at classifying fine-grained categories. This paper introduces a Retrieving And Ranking augmented method for MLLMs. Our proposed approach not only addresses the inherent limitations in fine-grained recognition but also preserves the model's comprehensive knowledge base.
  arXiv  Detail & Related papers  (2024-03-20T17:59:55Z)
• Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception [63.03288425612792]
  We propose bfAnyRef, a general MLLM model that can generate pixel-wise object perceptions and natural language descriptions from multi-modality references. Our model achieves state-of-the-art results across multiple benchmarks, including diverse modality referring segmentation and region-level referring expression generation.
  arXiv  Detail & Related papers  (2024-03-05T13:45:46Z)
• MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria [49.500322937449326]
  Multimodal large language models (MLLMs) have broadened the scope of AI applications. Existing automatic evaluation methodologies for MLLMs are mainly limited in evaluating queries without considering user experiences. We propose a new evaluation paradigm for MLLMs, which is evaluating MLLMs with per-sample criteria using potent MLLM as the judge.
  arXiv  Detail & Related papers  (2023-11-23T12:04:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.