BiasIG: Benchmarking Multi-dimensional Social Biases in Text-to-Image Models

• URL: http://arxiv.org/abs/2604.11934v1
• Date: Mon, 13 Apr 2026 18:22:50 GMT
• Title: BiasIG: Benchmarking Multi-dimensional Social Biases in Text-to-Image Models
• Abstract summary: We introduce BiasIG, a unified benchmark that quantifies social biases across a curated dataset of 47,040 prompts.<n>Extensive experiments on 8 T2I models and 3 debiasing methods validate BiasIG as a robust diagnostic tool.<n>Our work advocates for a precise, taxonomy-driven approach to fairness in AIGC, providing a theoretical framework for using BiasIG's metrics as feedback signals in future closed-loop mitigation.
• Score: 30.463069540851688
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-Image (T2I) generative models have revolutionized content creation, yet they inherently risk amplifying societal biases. While sociological research provides systematic classifications of bias, existing T2I benchmarks largely conflate these nuances or focus narrowly on occupational stereotypes, leaving the multi-dimensional nature of generative bias inadequately measured. In this paper, we introduce BiasIG, a unified benchmark that quantifies social biases across a curated dataset of 47,040 prompts. Grounded in sociological and machine ethics frameworks, BiasIG disentangles biases across 4 dimensions to enable fine-grained diagnosis. To facilitate scalable and reliable evaluation, we propose a fully automated pipeline powered by a fine-tuned multi-modal large language model, achieving high alignment accuracy comparable to human experts. Extensive experiments on 8 T2I models and 3 debiasing methods not only validate BiasIG as a robust diagnostic tool, but also reveal critical insights: interventions on protected attributes often trigger unintended confounding effects on unrelated demographics, and debiasing methods exhibit a persistent tendency toward discrimination rather than mere ignorance. Our work advocates for a precise, taxonomy-driven approach to fairness in AIGC, providing a theoretical framework for using BiasIG's metrics as feedback signals in future closed-loop mitigation. The benchmark is openly available at https://github.com/Astarojth/BiasIG.

Related papers

• Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement [66.67463557274358]
  Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes.<n>We introduce Confidence-guided Concept Mining (CBCM) to identify reliable spurious attributes without attribute annotations.<n>We propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches.
  arXiv  Detail & Related papers  (2026-06-23T05:28:47Z)
• HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing [20.795940054868854]
  Existing evaluation methods address only single-dimensional biases.<n>We introduce HoloFair, a comprehensive benchmark framework for multidimensional demographic bias analysis.<n>We also introduce Fair-GRPO, a reinforcement-learning-based debiasing method.
  arXiv  Detail & Related papers  (2026-05-23T17:52:07Z)
• Adaptive Generation of Bias-Eliciting Questions for LLMs [18.608477560948003]
  Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions worldwide.<n>We introduce a counterfactual bias evaluation framework that automatically generates realistic, open-ended questions over sensitive attributes such as sex, race, or religion.<n>We also capture distinct response dimensions that are increasingly relevant in user interactions, such as asymmetric refusals and explicit acknowledgment of bias.
  arXiv  Detail & Related papers  (2025-10-14T13:08:10Z)
• To Bias or Not to Bias: Detecting bias in News with bias-detector [1.8024397171920885]
  We perform sentence-level bias classification by fine-tuning a RoBERTa-based model on the expert-annotated BABE dataset.<n>We show statistically significant improvements in performance when comparing our model to a domain-adaptively pre-trained DA-RoBERTa baseline.<n>Our findings contribute to building more robust, explainable, and socially responsible NLP systems for media bias detection.
  arXiv  Detail & Related papers  (2025-05-19T11:54:39Z)
• Biased Heritage: How Datasets Shape Models in Facial Expression Recognition [13.77824359359967]
  We study bias propagation from datasets to trained models in image-based Facial Expression Recognition systems.<n>We introduce new bias metrics specifically designed for multiclass problems with multiple demographic groups.<n>Our findings suggest that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets.
  arXiv  Detail & Related papers  (2025-03-05T12:25:22Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Models [15.250980521130762]
  We introduce BIGbench, a unified benchmark for Biases of Image Generation.<n>Unlike existing benchmarks, BIGbench classifies and evaluates biases across four dimensions.<n>We apply BIGbench to evaluate eight representative T2I models and three debiasing methods.
  arXiv  Detail & Related papers  (2024-07-21T18:09:40Z)
• Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
  We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods.<n>We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results.<n>We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
  arXiv  Detail & Related papers  (2024-07-19T14:53:18Z)
• VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model [72.13121434085116]
  We introduce VLBiasBench, a benchmark to evaluate biases in Large Vision-Language Models (LVLMs)<n>VLBiasBench features a dataset that covers nine distinct categories of social biases, including age, disability status, gender, nationality, physical appearance, race, religion, profession, social economic status, as well as two intersectional bias categories: race x gender and race x social economic status.<n>We conduct extensive evaluations on 15 open-source models as well as two advanced closed-source models, yielding new insights into the biases present in these models.
  arXiv  Detail & Related papers  (2024-06-20T10:56:59Z)
• Quantifying Bias in Text-to-Image Generative Models [49.60774626839712]
  Bias in text-to-image (T2I) models can propagate unfair social representations and may be used to aggressively market ideas or push controversial agendas. Existing T2I model bias evaluation methods only focus on social biases. We propose an evaluation methodology to quantify general biases in T2I generative models, without any preconceived notions.
  arXiv  Detail & Related papers  (2023-12-20T14:26:54Z)
• Fast Model Debias with Machine Unlearning [54.32026474971696]
  Deep neural networks might behave in a biased manner in many real-world scenarios. Existing debiasing methods suffer from high costs in bias labeling or model re-training. We propose a fast model debiasing framework (FMD) which offers an efficient approach to identify, evaluate and remove biases.
  arXiv  Detail & Related papers  (2023-10-19T08:10:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.