Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts

• URL: http://arxiv.org/abs/2602.04398v1
• Date: Wed, 04 Feb 2026 10:27:36 GMT
• Title: Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts
• Authors: Yujie Lin, Kunquan Li, Yixuan Liao, Xiaoxin Chen, Jinsong Su,
• Abstract summary: We propose a framework for detecting stereotype-inducing words and attributing neuron-level bias in large language models.<n>Our framework first identifies stereotype-inducing adjectives and nouns via comparative analysis across demographic groups.<n> Experiments on three widely used LLMs demonstrate that our method effectively reduces bias while preserving overall model performance.
• Score: 29.864293711943038
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated impressive capabilities across a wide range of natural language processing tasks. However, their outputs often exhibit social biases, raising fairness concerns. Existing debiasing methods, such as fine-tuning on additional datasets or prompt engineering, face scalability issues or compromise user experience in multi-turn interactions. To address these challenges, we propose a framework for detecting stereotype-inducing words and attributing neuron-level bias in LLMs, without the need for fine-tuning or prompt modification. Our framework first identifies stereotype-inducing adjectives and nouns via comparative analysis across demographic groups. We then attribute biased behavior to specific neurons using two attribution strategies based on integrated gradients. Finally, we mitigate bias by directly intervening on their activations at the projection layer. Experiments on three widely used LLMs demonstrate that our method effectively reduces bias while preserving overall model performance. Code is available at the github link: https://github.com/XMUDeepLIT/Bi-directional-Bias-Attribution.

Related papers

• KnowBias: Mitigating Social Bias in LLMs via Know-Bias Neuron Enhancement [5.243877326529689]
  Large language models (LLMs) exhibit social biases that reinforce harmful stereotypes, limiting their safe deployment.<n>We propose KnowBias, a framework that mitigates bias by strengthening, rather than suppressing, neurons encoding bias-knowledge.<n>KnowBias identifies neurons encoding bias knowledge using a small set of bias-knowledge questions via attribution-based analysis, and selectively enhances them at inference time.
  arXiv  Detail & Related papers  (2026-01-29T15:32:38Z)
• Addressing Stereotypes in Large Language Models: A Critical Examination and Mitigation [0.0]
  Large Language models (LLMs) have gained popularity in recent years with the advancement of Natural Language Processing (NLP)<n>This study inspects and highlights the need to address biases in LLMs amid growing generative Artificial Intelligence (AI)<n>We utilize bias-specific benchmarks such StereoSet and CrowSPairs to evaluate the existence of various biases in many different generative models such as BERT, GPT 3.5, and ADA.
  arXiv  Detail & Related papers  (2025-11-18T05:43:34Z)
• A Comprehensive Study of Implicit and Explicit Biases in Large Language Models [1.0555164678638427]
  This study highlights the need to address biases in Large Language Models amid growing generative AI.<n>We studied bias-specific benchmarks such as StereoSet and CrowSPairs to evaluate the existence of various biases in multiple generative models such as BERT and GPT 3.5.<n>Results indicated fine-tuned models struggle with gender biases but excelled at identifying and avoiding racial biases.
  arXiv  Detail & Related papers  (2025-11-18T05:27:17Z)
• Adaptive Generation of Bias-Eliciting Questions for LLMs [18.608477560948003]
  Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions worldwide.<n>We introduce a counterfactual bias evaluation framework that automatically generates realistic, open-ended questions over sensitive attributes such as sex, race, or religion.<n>We also capture distinct response dimensions that are increasingly relevant in user interactions, such as asymmetric refusals and explicit acknowledgment of bias.
  arXiv  Detail & Related papers  (2025-10-14T13:08:10Z)
• Towards Resource Efficient and Interpretable Bias Mitigation in Large Language Models [1.787433808079955]
  Large language models (LLMs) have been observed to perpetuate unwanted biases in training data.<n>In this paper, we mitigate bias by leveraging small biased and anti-biased expert models to obtain a debiasing signal.<n> Experiments on mitigating gender, race, and religion biases show a reduction in bias on several local and global bias metrics.
  arXiv  Detail & Related papers  (2024-12-02T16:56:08Z)
• Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes [73.12947922129261]
  We leverage the zero-shot capabilities of large language models to reduce stereotyping. We show that self-debiasing can significantly reduce the degree of stereotyping across nine different social groups. We hope this work opens inquiry into other zero-shot techniques for bias mitigation.
  arXiv  Detail & Related papers  (2024-02-03T01:40:11Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• Mitigating Biases for Instruction-following Language Models via Bias Neurons Elimination [54.865941973768905]
  We propose a novel and practical bias mitigation method, CRISPR, to eliminate bias neurons of language models in instruction-following settings. CRISPR automatically determines biased outputs and categorizes neurons that affect the biased outputs as bias neurons using an explainability method. Experimental results demonstrate the effectiveness of our method in mitigating biases under zero-shot instruction-following settings without losing the model's task performance and existing knowledge.
  arXiv  Detail & Related papers  (2023-11-16T07:16:55Z)
• A Simple yet Effective Self-Debiasing Framework for Transformer Models [49.09053367249642]
  Current Transformer-based natural language understanding (NLU) models heavily rely on dataset biases. We propose a simple yet effective self-debiasing framework for Transformer-based NLU models.
  arXiv  Detail & Related papers  (2023-06-02T20:31:58Z)
• Debiasing Vision-Language Models via Biased Prompts [79.04467131711775]
  We propose a general approach for debiasing vision-language foundation models by projecting out biased directions in the text embedding. We show that debiasing only the text embedding with a calibrated projection matrix suffices to yield robust classifiers and fair generative models.
  arXiv  Detail & Related papers  (2023-01-31T20:09:33Z)
• Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks [76.35271072704384]
  Deep learning models perform poorly when applied to videos with rare scenes or objects. We tackle this problem from two different angles: algorithm and dataset. We show that the debiased representation can generalize better when transferred to other datasets and tasks.
  arXiv  Detail & Related papers  (2022-09-20T00:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.