Can Small-Scale Data Poisoning Exacerbate Dialect-Linked Biases in Large Language Models?

• URL: http://arxiv.org/abs/2507.19195v2
• Date: Thu, 09 Oct 2025 13:58:03 GMT
• Title: Can Small-Scale Data Poisoning Exacerbate Dialect-Linked Biases in Large Language Models?
• Authors: Chaymaa Abbas, Mariette Awad, Razane Tajeddine,
• Abstract summary: Style-conditioned data poisoning is identified as a covert vector for amplifying sociolinguistic bias.<n>This work probes whether linguistic style can act as a latent trigger for harmful behavior.
• Score: 2.7502681179700694
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Style-conditioned data poisoning is identified as a covert vector for amplifying sociolinguistic bias in large language models. Using small poisoned budgets that pair dialectal prompts -- principally African American Vernacular English (AAVE) and a Southern dialect -- with toxic or stereotyped completions during instruction tuning, this work probes whether linguistic style can act as a latent trigger for harmful behavior. Across multiple model families and scales, poisoned exposure elevates toxicity and stereotype expression for dialectal inputs -- most consistently for AAVE -- while Standard American English remains comparatively lower yet not immune. A multi-metric audit combining classifier-based toxicity with an LLM-as-a-judge reveals stereotype-laden content even when lexical toxicity appears muted, indicating that conventional detectors under-estimate sociolinguistic harms. Additionally, poisoned models exhibit emergent jailbreaking despite the absence of explicit slurs in the poison, suggesting weakened alignment rather than memorization. These findings underscore the need for dialect-aware evaluation, content-level stereotype auditing, and training protocols that explicitly decouple style from toxicity to prevent bias amplification through seemingly minor, style-based contamination.

Related papers

• Concept-Based Interpretability for Toxicity Detection [3.66086430438787]
  disproportionate attribution of concepts towards the target class often results in classification errors.<n>Our work introduces an interpretability technique based on the Concept Gradient (CG) method.<n>We propose the curation of toxic words that contribute to misclassifications in text classification models.
  arXiv  Detail & Related papers  (2025-11-15T14:53:23Z)
• Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models [3.3408746880885003]
  Large language models (LLMs) have shown promise, but they may rely on superficial cues leading to spurious predictions.<n>We demonstrate that mentions of alcohol or smoking can falsely induce models to predict current/past drug use where none is present.<n>We evaluate mitigation strategies - such as prompt engineering and chain-of-thought reasoning - to reduce these false positives.
  arXiv  Detail & Related papers  (2025-05-30T18:11:33Z)
• Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation [6.781972039785424]
  Recent generative large language models (LLMs) show remarkable performance in non-English languages.<n>When prompted in those languages they tend to express higher harmful social biases and toxicity levels.<n>We investigate the impact of different finetuning methods on the model's bias and toxicity, but also on its ability to produce fluent and diverse text.
  arXiv  Detail & Related papers  (2024-12-18T17:05:08Z)
• Leveraging Large Language Models and Topic Modeling for Toxicity Classification [2.1506858566021037]
  We investigate the impact of annotator positionality on the dataset while using topic-modeling strategies for content moderation.<n>Results indicate that fine-tuning the models on specific topics results in a notable improvement in the F1 score of the models.
  arXiv  Detail & Related papers  (2024-11-26T20:47:24Z)
• PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoning [32.508939142492004]
  We introduce PoisonBench, a benchmark for evaluating large language models' susceptibility to data poisoning during preference learning.<n>Data poisoning attacks can manipulate large language model responses to include hidden malicious content or biases.<n>We deploy two distinct attack types across eight realistic scenarios, assessing 21 widely-used models.
  arXiv  Detail & Related papers  (2024-10-11T13:50:50Z)
• Unlearnable Examples Detection via Iterative Filtering [84.59070204221366]
  Deep neural networks are proven to be vulnerable to data poisoning attacks. It is quite beneficial and challenging to detect poisoned samples from a mixed dataset. We propose an Iterative Filtering approach for UEs identification.
  arXiv  Detail & Related papers  (2024-08-15T13:26:13Z)
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Models [78.69526166193236]
  Pre-trained Language models (PLMs) have been acknowledged to contain harmful information, such as social biases. We propose sc Social Bias Neurons to accurately pinpoint units (i.e., neurons) in a language model that can be attributed to undesirable behavior, such as social bias. As measured by prior metrics from StereoSet, our model achieves a higher degree of fairness while maintaining language modeling ability with low cost.
  arXiv  Detail & Related papers  (2024-06-14T15:41:06Z)
• Unveiling the Implicit Toxicity in Large Language Models [77.90933074675543]
  The open-endedness of large language models (LLMs) combined with their impressive capabilities may lead to new safety issues when being exploited for malicious use. We show that LLMs can generate diverse implicit toxic outputs that are exceptionally difficult to detect via simply zero-shot prompting. We propose a reinforcement learning (RL) based attacking method to further induce the implicit toxicity in LLMs.
  arXiv  Detail & Related papers  (2023-11-29T06:42:36Z)
• CausalDialogue: Modeling Utterance-level Causality in Conversations [83.03604651485327]
  We have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing. This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure. We propose a causality-enhanced method called Exponential Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models.
  arXiv  Detail & Related papers  (2022-12-20T18:31:50Z)
• Toxicity Detection with Generative Prompt-based Inference [3.9741109244650823]
  It is a long-known risk that language models (LMs), once trained on corpus containing undesirable content, have the power to manifest biases and toxicity. In this work, we explore the generative variant of zero-shot prompt-based toxicity detection with comprehensive trials on prompt engineering.
  arXiv  Detail & Related papers  (2022-05-24T22:44:43Z)
• Cyberbullying Classifiers are Sensitive to Model-Agnostic Perturbations [15.152559543181523]
  This study is the first to investigate the effect of adversarial behavior and augmentation for cyberbullying detection. We demonstrate that model-agnostic lexical substitutions significantly hurt performance. Augmentations proposed in prior work on toxicity prove to be less effective.
  arXiv  Detail & Related papers  (2022-01-17T12:48:27Z)
• Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection [75.54119209776894]
  We investigate the effect of annotator identities (who) and beliefs (why) on toxic language annotations. We consider posts with three characteristics: anti-Black language, African American English dialect, and vulgarity. Our results show strong associations between annotator identity and beliefs and their ratings of toxicity.
  arXiv  Detail & Related papers  (2021-11-15T18:58:20Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Mitigating Biases in Toxic Language Detection through Invariant Rationalization [70.36701068616367]
  biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. We propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns. Our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.
  arXiv  Detail & Related papers  (2021-06-14T08:49:52Z)
• Challenges in Automated Debiasing for Toxic Language Detection [81.04406231100323]
  Biased associations have been a challenge in the development of classifiers for detecting toxic language. We investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English)
  arXiv  Detail & Related papers  (2021-01-29T22:03:17Z)
• RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models [93.151822563361]
  Pretrained neural language models (LMs) are prone to generating racist, sexist, or otherwise toxic language which hinders their safe deployment. We investigate the extent to which pretrained LMs can be prompted to generate toxic language, and the effectiveness of controllable text generation algorithms at preventing such toxic degeneration.
  arXiv  Detail & Related papers  (2020-09-24T03:17:19Z)
• Limits of Detecting Text Generated by Large-Scale Language Models [65.46403462928319]
  Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns. Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated.
  arXiv  Detail & Related papers  (2020-02-09T19:53:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.