A Representation-Level Assessment of Bias Mitigation in Foundation Models

• URL: http://arxiv.org/abs/2604.08561v1
• Date: Tue, 17 Mar 2026 12:43:52 GMT
• Title: A Representation-Level Assessment of Bias Mitigation in Foundation Models
• Abstract summary: We investigate how successful bias mitigation reshapes the embedding space of encoder-only and decoder-only foundation models.<n>Our findings show that bias mitigation reduces gender-occupation disparities in the embedding space.<n>To further promote the assessment of decoder-only models, we introduce WinoDec, a dataset consisting of 4,000 sequences with gender and occupation terms.
• Score: 14.111258398730273
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We investigate how successful bias mitigation reshapes the embedding space of encoder-only and decoder-only foundation models, offering an internal audit of model behaviour through representational analysis. Using BERT and Llama2 as representative architectures, we assess the shifts in associations between gender and occupation terms by comparing baseline and bias-mitigated variants of the models. Our findings show that bias mitigation reduces gender-occupation disparities in the embedding space, leading to more neutral and balanced internal representations. These representational shifts are consistent across both model types, suggesting that fairness improvements can manifest as interpretable and geometric transformations. These results position embedding analysis as a valuable tool for understanding and validating the effectiveness of debiasing methods in foundation models. To further promote the assessment of decoder-only models, we introduce WinoDec, a dataset consisting of 4,000 sequences with gender and occupation terms, and release it to the general public. (https://github.com/winodec/wino-dec)

Related papers

• DualEval: Joint Model-Item Calibration for Unified LLM Evaluation [67.99119926012686]
  DualEval is a latent model-item calibration framework that represents models and evaluation items in a shared space.<n>We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries.
  arXiv  Detail & Related papers  (2026-06-24T22:40:46Z)
• Classifier Reconstruction Through Counterfactual-Aware Wasserstein Prototypes [7.568155070224663]
  We demonstrate that model reconstruction can be significantly improved by recognizing that counterfactuals can serve as informative though less representative samples for both classes.<n>We propose a method that integrates original data samples with counterfactuals to approximate class prototypes using the Wasserstein barycenter.
  arXiv  Detail & Related papers  (2025-12-11T18:06:49Z)
• Decomposing Direct and Indirect Biases in Linear Models under Demographic Parity Constraint [4.129225533930966]
  We propose a post-processing framework to decompose the resulting bias into direct (sensitive-attribute) and indirect (correlated-features) components.<n>Our method analytically characterizes how demographic parity reshapes each model coefficient, including those of both sensitive and non-sensitive features.<n>Our framework requires no retraining and provides actionable insights for model auditing and mitigation.
  arXiv  Detail & Related papers  (2025-11-14T13:27:54Z)
• Towards a Unified Representation Evaluation Framework Beyond Downstream Tasks [9.390951257874187]
  We introduce a standardized protocol to quantify informativeness, equivariance, invariance, and disentanglement of factors of variation in model representations.<n>We find that representations from models with similar downstream performance can behave substantially differently with regard to these attributes.
  arXiv  Detail & Related papers  (2025-05-09T17:58:52Z)
• Rethinking Relation Extraction: Beyond Shortcuts to Generalization with a Debiased Benchmark [53.876493664396506]
  Benchmarks are crucial for evaluating machine learning algorithm performance, facilitating comparison and identifying superior solutions.<n>This paper addresses the issue of entity bias in relation extraction tasks, where models tend to rely on entity mentions rather than context.<n>We propose a debiased relation extraction benchmark DREB that breaks the pseudo-correlation between entity mentions and relation types through entity replacement.<n>To establish a new baseline on DREB, we introduce MixDebias, a debiasing method combining data-level and model training-level techniques.
  arXiv  Detail & Related papers  (2025-01-02T17:01:06Z)
• Comparing Fairness of Generative Mobility Models [3.699135947901772]
  This work examines the fairness of generative mobility models, addressing the often overlooked dimension of equity in model performance across geographic regions. Predictive models built on crowd flow data are instrumental in understanding urban structures and movement patterns. We propose a novel framework for assessing fairness by measuring utility and equity of generated traces.
  arXiv  Detail & Related papers  (2024-11-07T06:01:12Z)
• DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision [73.80009454050858]
  This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Our model jointly optimize for two fairness criteria - group fairness and counterfactual fairness.
  arXiv  Detail & Related papers  (2023-03-15T07:13:54Z)
• The Unbearable Weight of Massive Privilege: Revisiting Bias-Variance Trade-Offs in the Context of Fair Prediction [7.975779552420981]
  We propose a conditional-iid (ciid) model that seeks to improve on the trade-offs made by a single model. We empirically test our setup on the COMPAS and folktables datasets. Our analysis suggests that there might be principled procedures and concrete real-world use cases under which conditional models are preferred.
  arXiv  Detail & Related papers  (2023-02-17T05:34:35Z)
• Debiasing Vision-Language Models via Biased Prompts [79.04467131711775]
  We propose a general approach for debiasing vision-language foundation models by projecting out biased directions in the text embedding. We show that debiasing only the text embedding with a calibrated projection matrix suffices to yield robust classifiers and fair generative models.
  arXiv  Detail & Related papers  (2023-01-31T20:09:33Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.