CIFA: Contextual-Intersectional Fairness Auditing for Hidden Subgroup Discovery in Face Analysis

• URL: http://arxiv.org/abs/2608.09669v1
• Date: Mon, 10 Aug 2026 14:44:03 GMT
• Title: CIFA: Contextual-Intersectional Fairness Auditing for Hidden Subgroup Discovery in Face Analysis
• Abstract summary: We propose the Contextual-Intersectional Fairness Auditing Framework (CIFA) for identifying subgroup vulnerabilities.<n>CIFA performs demographic, contextual, and contextual-intersectional auditing, followed by worst-group discovery to identify and rank the most vulnerable attribute combinations.<n>Our results show that aggregate accuracy and demographic-only evaluation can mask substantial contextual-intersectional disparities.
• Score: 24.75673745723537
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fairness evaluation in computer vision commonly relies on aggregate accuracy and demographic subgroup analysis. However, visual models are also sensitive to contextual factors such as illumination, blur, image quality, facial accessories, and appearance attributes. These factors may interact with demographic characteristics, producing hidden subgroups in which performance degrades substantially despite strong aggregate accuracy and apparently acceptable demographic fairness. To address this, we propose the Contextual-Intersectional Fairness Auditing Framework (CIFA), a structured framework for identifying subgroup vulnerabilities arising from interactions between demographic and contextual attributes. CIFA performs demographic, contextual, and contextual-intersectional auditing, followed by worst-group discovery to identify and rank the most vulnerable attribute combinations. We evaluate CIFA on gender classification using ResNet-50 \cite{he2016deep} and ViT-B/16 \cite{dosovitskiy2020image} across FairFace \cite{Karkkainen2021}, CelebA \cite{Liu2015}, and UTKFace \cite{Zhang2017}. Our results show that aggregate accuracy and demographic-only evaluation can mask substantial contextual-intersectional disparities. We further assess several established mitigation strategies through an audit--mitigate--reaudit protocol and find that, although some worst-group disparities are reduced, no single strategy consistently eliminates them across datasets and architectures. These findings establish contextual-intersectional auditing as an important component of fairness evaluation and provide a reproducible framework for discovering, prioritizing, and reassessing hidden subgroup risks in face analysis systems.

Related papers

• WIDER-FAIR: An Annotated Version of the WIDER-FACE Dataset for Fairness Evaluation [0.7034739490820968]
  We introduce a new face detection dataset built on the widely used WIDER-FACE benchmark, manually annotated with the perceived ethnicity and sex of each face.<n>The dataset contains 16,256 images annotated across four ethnic groups: Asian, Black, Indian, and White, and two sex categories.<n>We show that detection performance is notably lower for faces of Black individuals, and that excluding this group from training increases fairness disparity more than excluding any other ethnic group.
  arXiv  Detail & Related papers  (2026-06-30T14:10:42Z)
• Mitigating Bias with Words: Inducing Demographic Ambiguity in Face Recognition Templates by Text Encoding [18.08946802592489]
  Face recognition systems are often prone to demographic biases, partially due to the entanglement of demographic-specific information with identity-relevant features in facial embeddings.<n>We propose a novel strategy, Unified Text-Image Embedding (UTIE), which aims to induce demographic ambiguity in face embeddings.<n>We show that UTIE consistently reduces bias metrics while maintaining, or even improving in several cases, the face verification accuracy.
  arXiv  Detail & Related papers  (2025-12-05T10:41:58Z)
• Latent Feature Alignment: Discovering Biased and Interpretable Subpopulations in Face Recognition Models [2.098253213160396]
  Latent Feature Alignment (LFA) is an attribute-label-free algorithm that uses latent directions to identify subpopulations.<n>LFA consistently outperforms k-means and nearest-neighbor search in intra-group semantic coherence.<n>These results position LFA as a practical method for representation auditing of face recognition models.
  arXiv  Detail & Related papers  (2025-10-17T10:49:50Z)
• Fair Deepfake Detectors Can Generalize [51.21167546843708]
  We show that controlling for confounders (data distribution and model capacity) enables improved generalization via fairness interventions.<n>Motivated by this insight, we propose Demographic Attribute-insensitive Intervention Detection (DAID), a plug-and-play framework composed of: i) Demographic-aware data rebalancing, which employs inverse-propensity weighting and subgroup-wise feature normalization to neutralize distributional biases; and ii) Demographic-agnostic feature aggregation, which uses a novel alignment loss to suppress sensitive-attribute signals.<n>DAID consistently achieves superior performance in both fairness and generalization compared to several state-of-the-art
  arXiv  Detail & Related papers  (2025-07-03T14:10:02Z)
• $C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction [80.57232374640911]
  We propose a model-agnostic strategy called the Mask-And-Recover (MAR)<n>MAR integrates both inter- and intra-modality contextual correlations to enable global inference within extraction modules.<n>To better target challenging parts within each sample, we introduce a Fine-grained Confidence Score (FCS) model.
  arXiv  Detail & Related papers  (2025-04-01T13:01:30Z)
• FairSAM: Fair Classification on Corrupted Data Through Sharpness-Aware Minimization [12.178322948983263]
  Image classification models trained on clean data often suffer from significant performance degradation when exposed to testing corrupted data.<n>This degradation not only impacts overall performance but also disproportionately affects various demographic subgroups, raising critical algorithmic bias concerns.<n>Existing fairness-aware machine learning methods aim to reduce performance disparities but hardly maintain robust and equitable accuracy when faced with data corruption.<n>We propose textbfFairSAM, a new framework that integrates underlineFairness-oriented strategies into underlineSAM to deliver equalized performance across demographic groups under corrupted conditions.
  arXiv  Detail & Related papers  (2025-03-29T01:51:59Z)
• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics [80.07271410743806]
  We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
  arXiv  Detail & Related papers  (2023-03-25T09:34:05Z)
• Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
  There are demographic biases present in current facial recognition (FR) models. We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups. We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
  arXiv  Detail & Related papers  (2021-03-16T15:05:49Z)
• Mitigating Face Recognition Bias via Group Adaptive Classifier [53.15616844833305]
  This work aims to learn a fair face representation, where faces of every group could be more equally represented. Our work is able to mitigate face recognition bias across demographic groups while maintaining the competitive accuracy.
  arXiv  Detail & Related papers  (2020-06-13T06:43:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.