Can We Locate and Prevent Stereotypes in LLMs?

• URL: http://arxiv.org/abs/2604.19764v1
• Date: Thu, 26 Mar 2026 20:59:08 GMT
• Title: Can We Locate and Prevent Stereotypes in LLMs?
• Abstract summary: This study investigates the internal mechanisms of GPT 2 Small and Llama 3.2 to locate stereotype related activations.<n>We identify individual contrastive neuron activations that encode stereotypes, and detecting attention heads that contribute heavily to biased outputs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stereotypes in large language models (LLMs) can perpetuate harmful societal biases. Despite the widespread use of models, little is known about where these biases reside in the neural network. This study investigates the internal mechanisms of GPT 2 Small and Llama 3.2 to locate stereotype related activations. We explore two approaches: identifying individual contrastive neuron activations that encode stereotypes, and detecting attention heads that contribute heavily to biased outputs. Our experiments aim to map these "bias fingerprints" and provide initial insights for mitigating stereotypes.

Related papers

• H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs [56.31565301428888]
  We identify hallucination-associated neurons (H-Neurons) in large language models (LLMs)<n>In terms of identification, we demonstrate that a remarkably sparse subset of neurons can reliably predict hallucination occurrences.<n>In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors.
  arXiv  Detail & Related papers  (2025-12-01T15:32:14Z)
• StereoDetect: Detecting Stereotypes and Anti-stereotypes the Correct Way Using Social Psychological Underpinnings [47.02959423049043]
  Stereotype and Anti-stereotype detection is a problem that requires social knowledge.<n>We propose a five-tuple definition and provide precise terminologies disentangling stereotypes, anti-stereotypes, stereotypical bias, and general bias.<n>We show that sub-10B language models and GPT-4o frequently misclassify anti-stereotypes and fail to recognize neutral overgeneralizations.
  arXiv  Detail & Related papers  (2025-04-04T11:14:38Z)
• $\texttt{ModSCAN}$: Measuring Stereotypical Bias in Large Vision-Language Models from Vision and Language Modalities [30.960327354387054]
  Large vision-language models (LVLMs) have been rapidly developed and widely used in various fields, but the (potential) stereotypical bias in the model is largely unexplored. We present a pioneering measurement framework, $textttModSCAN$, to $underlineSCAN$ the stereotypical bias within LVLMs.
  arXiv  Detail & Related papers  (2024-10-09T15:07:05Z)
• Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models [9.734705470760511]
  We use GlobalBias to study a broad set of stereotypes from around the world. We generate character profiles based on given names and evaluate the prevalence of stereotypes in model outputs.
  arXiv  Detail & Related papers  (2024-07-09T14:52:52Z)
• MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries? [70.77691645678804]
  Humans are prone to cognitive distortions -- biased thinking patterns that lead to exaggerated responses to specific stimuli. This paper demonstrates that advanced Multimodal Large Language Models (MLLMs) exhibit similar tendencies. We identify three types of stimuli that trigger the oversensitivity of existing MLLMs: Exaggerated Risk, Negated Harm, and Counterintuitive.
  arXiv  Detail & Related papers  (2024-06-22T23:26:07Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach [4.908389661988191]
  This paper introduces the Multi-Grain Stereotype (MGS) dataset, consisting of 51,867 instances across gender, race, profession, religion, and other stereotypes. We evaluate various machine learning approaches to establish baselines and fine-tune language models of different architectures and sizes. We employ explainable AI (XAI) tools, including SHAP, LIME, and BertViz, to assess whether the model's learned patterns align with human intuitions about stereotypes.
  arXiv  Detail & Related papers  (2024-04-02T09:31:32Z)
• Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes [73.12947922129261]
  We leverage the zero-shot capabilities of large language models to reduce stereotyping. We show that self-debiasing can significantly reduce the degree of stereotyping across nine different social groups. We hope this work opens inquiry into other zero-shot techniques for bias mitigation.
  arXiv  Detail & Related papers  (2024-02-03T01:40:11Z)
• Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale [61.555788332182395]
  We investigate the potential for machine learning models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects.
  arXiv  Detail & Related papers  (2022-11-07T18:31:07Z)
• Understanding and Countering Stereotypes: A Computational Approach to the Stereotype Content Model [4.916009028580767]
  We present a computational approach to interpreting stereotypes in text through the Stereotype Content Model (SCM) The SCM proposes that stereotypes can be understood along two primary dimensions: warmth and competence. It is known that countering stereotypes with anti-stereotypical examples is one of the most effective ways to reduce biased thinking.
  arXiv  Detail & Related papers  (2021-06-04T16:53:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.