BioPro: On Difference-Aware Gender Fairness for Vision-Language Models

• URL: http://arxiv.org/abs/2512.00807v1
• Date: Sun, 30 Nov 2025 09:33:09 GMT
• Title: BioPro: On Difference-Aware Gender Fairness for Vision-Language Models
• Authors: Yujie Lin, Jiayao Ma, Qingguo Hu, Derek F. Wong, Jinsong Su,
• Abstract summary: Vision-Language Models (VLMs) inherit significant social biases from their training data, notably in gender representation.<n>We build upon recent advances in difference-aware fairness for text-only models to formalize the problem of difference-aware gender fairness for image captioning and text-to-image generation.<n>We propose BioPro, which aims to mitigate unwanted bias in neutral contexts while preserving valid distinctions in explicit ones.
• Score: 50.40913324046528
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Models (VLMs) inherit significant social biases from their training data, notably in gender representation. Current fairness interventions often adopt a difference-unaware perspective that enforces uniform treatment across demographic groups. These approaches, however, fail to distinguish between contexts where neutrality is required and those where group-specific attributes are legitimate and must be preserved. Building upon recent advances in difference-aware fairness for text-only models, we extend this concept to the multimodal domain and formalize the problem of difference-aware gender fairness for image captioning and text-to-image generation. We advocate for selective debiasing, which aims to mitigate unwanted bias in neutral contexts while preserving valid distinctions in explicit ones. To achieve this, we propose BioPro (Bias Orthogonal Projection), an entirely training-free framework. BioPro identifies a low-dimensional gender-variation subspace through counterfactual embeddings and applies projection to selectively neutralize gender-related information. Experiments show that BioPro effectively reduces gender bias in neutral cases while maintaining gender faithfulness in explicit ones, thus providing a promising direction toward achieving selective fairness in VLMs. Beyond gender bias, we further demonstrate that BioPro can effectively generalize to continuous bias variables, such as scene brightness, highlighting its broader applicability.

Related papers

• More of the Same: Persistent Representational Harms Under Increased Representation [12.071592182704707]
  We develop an evaluation methodology for surfacing distribution-level group representational biases in generated text.<n>We show that, even though the gender distribution when models are prompted to generate biographies leads to a large representation of women, even representational biases persist in how different genders are represented.
  arXiv  Detail & Related papers  (2025-03-01T03:45:35Z)
• The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models [91.86718720024825]
  We center transgender, nonbinary, and other gender-diverse identities to investigate how alignment procedures interact with pre-existing gender-diverse bias.<n>Our findings reveal that DPO-aligned models are particularly sensitive to supervised finetuning.<n>We conclude with recommendations tailored to DPO and broader alignment practices.
  arXiv  Detail & Related papers  (2024-11-06T06:50:50Z)
• Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Words [85.48043537327258]
  Existing machine translation gender bias evaluations are primarily focused on male and female genders. This study presents a benchmark AmbGIMT (Gender-Inclusive Machine Translation with Ambiguous attitude words) We propose a novel process to evaluate gender bias based on the Emotional Attitude Score (EAS), which is used to quantify ambiguous attitude words.
  arXiv  Detail & Related papers  (2024-07-23T08:13:51Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias [23.10522891268232]
  We introduce a Mixture-of-Experts approach to mitigate gender bias in text-to-image models. We show that our approach successfully mitigates gender bias while maintaining image quality.
  arXiv  Detail & Related papers  (2024-06-25T14:59:31Z)
• VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution [80.57383975987676]
  VisoGender is a novel dataset for benchmarking gender bias in vision-language models. We focus on occupation-related biases within a hegemonic system of binary gender, inspired by Winograd and Winogender schemas. We benchmark several state-of-the-art vision-language models and find that they demonstrate bias in resolving binary gender in complex scenes.
  arXiv  Detail & Related papers  (2023-06-21T17:59:51Z)
• Social Norm Bias: Residual Harms of Fairness-Aware Algorithms [21.50551404445654]
  Social Norm Bias (SNoB) is a subtle but consequential type of discrimination that may be exhibited by automated decision-making systems. We quantify SNoB by measuring how an algorithm's predictions are associated with conformity to gender norms. We show that post-processing interventions do not mitigate this type of bias at all.
  arXiv  Detail & Related papers  (2021-08-25T05:54:56Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.