Face Identification Proficiency Test Designed Using Item Response Theory

• URL: http://arxiv.org/abs/2106.15323v2
• Date: Thu, 1 Jul 2021 16:52:11 GMT
• Title: Face Identification Proficiency Test Designed Using Item Response Theory
• Abstract summary: Triad Identity Matching (TIM) test is based on stimulus difficulty measures based on Item Response Theory (IRT) In Experiment 1, university students showed wide-ranging accuracy on the TIM test. IRT-based item difficulty measures were used to partition the TIM test into three equally "easy" and three equally "difficult" subsets.
• Score: 5.284353550094401
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Measures of face identification proficiency are essential to ensure accurate and consistent performance by professional forensic face examiners and others who perform face identification tasks in applied scenarios. Current proficiency tests rely on static sets of stimulus items, and so, cannot be administered validly to the same individual multiple times. To create a proficiency test, a large number of items of "known" difficulty must be assembled. Multiple tests of equal difficulty can be constructed then using subsets of items. Here, we introduce a proficiency test, the Triad Identity Matching (TIM) test, based on stimulus difficulty measures based on Item Response Theory (IRT). Participants view face-image "triads" (N=225) (two images of one identity and one image of a different identity) and select the different identity. In Experiment 1, university students (N=197) showed wide-ranging accuracy on the TIM test. Furthermore, IRT modeling demonstrated that the TIM test produces items of various difficulty levels. In Experiment 2, IRT-based item difficulty measures were used to partition the TIM test into three equally "easy" and three equally "difficult" subsets. Simulation results indicated that the full set, as well as curated subsets, of the TIM items yielded reliable estimates of subject ability. In summary, the TIM test can provide a starting point for developing a framework that is flexible, calibrated, and adaptive to measure proficiency across various ability levels (e.g., professionals or populations with face processing deficits)

Related papers

• TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement [7.122068644799895]
  Test-time Training enables model adaptation using only test questions.<n>We propose textbfTTSR, a self-reflective test-time self-evolving training framework.
  arXiv  Detail & Related papers  (2026-02-06T18:55:40Z)
• Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning [112.16686518063456]
  We introduce textbfMulti-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time.<n>MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making.<n>Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67% over a multi-agent baseline, and by 8.67% over comparable single-agent baselines
  arXiv  Detail & Related papers  (2026-01-14T17:57:43Z)
• DRIVE-T: A Methodology for Discriminative and Representative Data Viz Item Selection for Literacy Construct and Assessment [1.9389881806157316]
  DRIVE-T is a methodology designed to drive the construction and evaluation of assessment items.<n>It supports the identification of task-based items discriminability and representativeness for measuring levels of data visualization literacy.<n>We show and apply each step of the methodology to an item bank, which models the difficulty levels of a measurement construct.
  arXiv  Detail & Related papers  (2025-08-06T07:34:40Z)
• Sample Complexity and Representation Ability of Test-time Scaling Paradigms [91.34339030453425]
  Test-time scaling paradigms have advanced the capabilities of large language models (LLMs) on complex tasks.<n>We study the sample efficiency of various test-time strategies, such as self-consistency, best-of-$n$, and self-correction.<n>A single Transformer architecture can provably solve multiple tasks without prior knowledge of the specific task associated with a user query.
  arXiv  Detail & Related papers  (2025-06-05T17:48:19Z)
• TestAgent: An Adaptive and Intelligent Expert for Human Assessment [62.060118490577366]
  We propose TestAgent, a large language model (LLM)-powered agent designed to enhance adaptive testing through interactive engagement.<n>TestAgent supports personalized question selection, captures test-takers' responses and anomalies, and provides precise outcomes through dynamic, conversational interactions.
  arXiv  Detail & Related papers  (2025-06-03T16:07:54Z)
• Goldilocks Test Sets for Face Verification [5.518961562474026]
  We propose three challenging test sets to reveal important but ignored weaknesses of the existing face recognition algorithms.<n>To challenge FR models on similar-looking persons, we propose twins-IND, which contains images from a dedicated twins dataset.<n>The proposed test sets have on-par or higher difficulties than the existing test sets.
  arXiv  Detail & Related papers  (2024-05-24T22:27:10Z)
• Identity Overlap Between Face Recognition Train/Test Data: Causing Optimistic Bias in Accuracy Measurement [5.3490692492497836]
  overlap between training and testing sets causes an optimistic accuracy estimate. Accuracy is commonly estimated as average 10-fold classification accuracy on image pairs from test sets. Our experiments reveal a surprising degree of identity and image overlap between the LFW family of test sets and the MS1MV2 training set.
  arXiv  Detail & Related papers  (2024-05-15T14:59:26Z)
• Task-adaptive Q-Face [75.15668556061772]
  We propose a novel task-adaptive multi-task face analysis method named as Q-Face. Q-Face simultaneously performs multiple face analysis tasks with a unified model. Our method achieves state-of-the-art performance on face expression recognition, action unit detection, face attribute analysis, age estimation, and face pose estimation.
  arXiv  Detail & Related papers  (2024-05-15T03:13:11Z)
• Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate Models for IRT Assessment [0.6138671548064356]
  We propose training pre-trained language models (PLMs) as surrogate models to enable item response theory (IRT) assessment. We also propose two strategies to control the difficulty levels of both the gaps and the distractors using ranking rules to reduce invalid distractors.
  arXiv  Detail & Related papers  (2024-03-03T09:18:05Z)
• ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbation [18.103478658038846]
  Text-to-Image (T2I) Diffusion Models (DMs) have shown impressive abilities in generating high-quality images based on simple text descriptions. As is common with many Deep Learning (DL) models, DMs are subject to a lack of robustness. We introduce a probabilistic notion of T2I DMs' robustness; and then establish an efficient framework, ProTIP, to evaluate it with statistical guarantees.
  arXiv  Detail & Related papers  (2024-02-23T16:48:56Z)
• SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models [63.946809247201905]
  We introduce a new benchmark, namely SHIELD, to evaluate the ability of MLLMs on face spoofing and forgery detection. We design true/false and multiple-choice questions to evaluate multimodal face data in these two face security tasks. The results indicate that MLLMs hold substantial potential in the face security domain.
  arXiv  Detail & Related papers  (2024-02-06T17:31:36Z)
• Modeling the Uncertainty with Maximum Discrepant Students for Semi-supervised 2D Pose Estimation [57.17120203327993]
  We propose a framework to estimate the quality of pseudo-labels in semi-supervised pose estimation tasks. Our method improves the performance of semi-supervised pose estimation on three datasets.
  arXiv  Detail & Related papers  (2023-11-03T08:11:06Z)
• ClusT3: Information Invariant Test-Time Training [19.461441044484427]
  Test-time training (TTT) methods have been developed in an attempt to mitigate these vulnerabilities. We propose a novel unsupervised TTT technique based on the Mutual of Mutual Information between multi-scale feature maps and a discrete latent representation. Experimental results demonstrate competitive classification performance on different popular test-time adaptation benchmarks.
  arXiv  Detail & Related papers  (2023-10-18T21:43:37Z)
• On Pitfalls of Test-Time Adaptation [82.8392232222119]
  Test-Time Adaptation (TTA) has emerged as a promising approach for tackling the robustness challenge under distribution shifts. We present TTAB, a test-time adaptation benchmark that encompasses ten state-of-the-art algorithms, a diverse array of distribution shifts, and two evaluation protocols.
  arXiv  Detail & Related papers  (2023-06-06T09:35:29Z)
• Unimodal Face Classification with Multimodal Training [1.9580473532948401]
  We propose a Multimodal Training Unimodal Test (MTUT) framework for robust face classification. Our framework exploits the cross-modality relationship during training and applies it as a complementary of the imperfect single modality input during testing. We experimentally show that our MTUT framework consistently outperforms ten baselines on 2D and 3D settings of both datasets.
  arXiv  Detail & Related papers  (2021-12-08T09:12:47Z)
• A New Score for Adaptive Tests in Bayesian and Credal Networks [64.80185026979883]
  A test is adaptive when its sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain.
  arXiv  Detail & Related papers  (2021-05-25T20:35:42Z)
• Measuring Massive Multitask Language Understanding [79.6985576698597]
  The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more. Largest GPT-3 model improves over random chance by almost 20 percentage points on average. Models also have lopsided performance and frequently do not know when they are wrong.
  arXiv  Detail & Related papers  (2020-09-07T17:59:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.