A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models

• URL: http://arxiv.org/abs/2608.05467v1
• Date: Wed, 05 Aug 2026 23:33:01 GMT
• Title: A Mechanistic Analysis of Gender Sensitivity in Dense Retrieval Models
• Abstract summary: We analyze bi-encoder models to localize gender sensitivity, finding that the signal originates in input embeddings and propagates through a small set of late-layer attention heads.<n>Our findings provide a mechanistic basis for targeted debiasing and highlight the challenge of disentangling gender from relevance signals in shared model components.
• Score: 68.90383874606164
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While gender bias in dense retrieval models is well documented, with prior work showing that models often score male-gendered documents higher than female or neutral variants, the internal mechanisms producing these disparities are poorly understood. In this paper, we mechanistically analyze bi-encoder models to localize gender sensitivity, finding that the signal originates in input embeddings and propagates through a small set of late-layer attention heads that carry both gender and term-matching signals. Guided by these findings, we test steering interventions at both identified points and find distinct effects: embedding-level steering non-specifically neutralizes score differences, while attention-level steering produces directional shifts. Our findings provide a mechanistic basis for targeted debiasing and highlight the challenge of disentangling gender from relevance signals in shared model components.

Related papers

• Mitigating Gender Bias in Depression Detection via Counterfactual Inference [16.49171032612706]
  Depression detection models often suffer from gender bias due to imbalanced training data.<n>We propose a novel Counterfactual Debiasing Framework grounded in causal inference.<n>Our framework not only significantly reduces gender bias but also improves overall detection performance.
  arXiv  Detail & Related papers  (2025-12-01T16:14:20Z)
• Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation [116.86965910589775]
  We show that even minimal perturbations, such as masking just 10% of objects or weakly blurring backgrounds, can dramatically alter bias scores.<n>This suggests that current bias evaluations reflect model responses to spurious features rather than gender bias.
  arXiv  Detail & Related papers  (2025-09-09T11:14:11Z)
• Fair Deepfake Detectors Can Generalize [51.21167546843708]
  We show that controlling for confounders (data distribution and model capacity) enables improved generalization via fairness interventions.<n>Motivated by this insight, we propose Demographic Attribute-insensitive Intervention Detection (DAID), a plug-and-play framework composed of: i) Demographic-aware data rebalancing, which employs inverse-propensity weighting and subgroup-wise feature normalization to neutralize distributional biases; and ii) Demographic-agnostic feature aggregation, which uses a novel alignment loss to suppress sensitive-attribute signals.<n>DAID consistently achieves superior performance in both fairness and generalization compared to several state-of-the-art
  arXiv  Detail & Related papers  (2025-07-03T14:10:02Z)
• Fairness in Machine Learning-based Hand Load Estimation: A Case Study on Load Carriage Tasks [1.1674893622721483]
  We developed and evaluated a fair predictive model for hand load estimation that leverages a Variational Autoencoder (VAE) with feature disentanglement.<n>Our proposed fair algorithm outperformed conventional machine learning methods in both fairness and predictive accuracy, achieving a lower mean absolute error (MAE) difference across male and female sets.<n>These findings emphasize the importance of fairness-aware machine learning algorithms to prevent potential disadvantages in workplace health and safety for certain worker populations.
  arXiv  Detail & Related papers  (2025-04-08T01:55:40Z)
• Gender Encoding Patterns in Pretrained Language Model Representations [17.101242741559428]
  Gender bias in pretrained language models (PLMs) poses significant social and ethical challenges.<n>This study adopts an information-theoretic approach to analyze how gender biases are encoded within various encoder-based architectures.
  arXiv  Detail & Related papers  (2025-03-09T19:17:46Z)
• Out-of-Distribution Detection on Graphs: A Survey [58.47395497985277]
  Graph out-of-distribution (GOOD) detection focuses on identifying graph data that deviates from the distribution seen during training.<n>We categorize existing methods into four types: enhancement-based, reconstruction-based, information propagation-based, and classification-based approaches.<n>We discuss practical applications and theoretical foundations, highlighting the unique challenges posed by graph data.
  arXiv  Detail & Related papers  (2025-02-12T04:07:12Z)
• Blind Men and the Elephant: Diverse Perspectives on Gender Stereotypes in Benchmark Datasets [12.798832545154271]
  This paper examines the inconsistencies between intrinsic stereotype benchmarks.<n>Using StereoSet and CrowS-Pairs as case studies, we investigated how data distribution affects benchmark results.
  arXiv  Detail & Related papers  (2025-01-02T09:40:31Z)
• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• Identifying and examining machine learning biases on Adult dataset [0.7856362837294112]
  This research delves into the reduction of machine learning model bias through Ensemble Learning. Our rigorous methodology comprehensively assesses bias across various categorical variables, ultimately revealing a pronounced gender attribute bias. This study underscores ethical considerations and advocates the implementation of hybrid models for a data-driven society marked by inclusivity and impartiality.
  arXiv  Detail & Related papers  (2023-10-13T19:41:47Z)
• Gender Biases in Automatic Evaluation Metrics for Image Captioning [87.15170977240643]
  We conduct a systematic study of gender biases in model-based evaluation metrics for image captioning tasks. We demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations. We present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments.
  arXiv  Detail & Related papers  (2023-05-24T04:27:40Z)
• Mitigating Gender Bias in Face Recognition Using the von Mises-Fisher Mixture Model [7.049738935364298]
  In this work, we investigate the gender bias of deep Face Recognition networks. Motivated by geometric considerations, we mitigate gender bias through a new post-processing methodology. In fact, extensive numerical experiments on a variety of datasets show that a careful selection significantly reduces gender bias.
  arXiv  Detail & Related papers  (2022-10-24T23:53:56Z)
• Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias [45.956112337250275]
  We propose a methodology grounded in the theory of causal mediation analysis for interpreting which parts of a model are causally implicated in its behavior. We apply this methodology to analyze gender bias in pre-trained Transformer language models. Our mediation analysis reveals that gender bias effects are (i) sparse, concentrated in a small part of the network; (ii) synergistic, amplified or repressed by different components; and (iii) decomposable into effects flowing directly from the input and indirectly through the mediators.
  arXiv  Detail & Related papers  (2020-04-26T01:53:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.