Bias Amplification: Large Language Models as Increasingly Biased Media

• URL: http://arxiv.org/abs/2410.15234v2
• Date: Mon, 17 Feb 2025 07:49:14 GMT
• Title: Bias Amplification: Large Language Models as Increasingly Biased Media
• Authors: Ze Wang, Zekun Wu, Jeremy Zhang, Xin Guan, Navya Jain, Skylar Lu, Saloni Gupta, Adriano Koshiyama,
• Abstract summary: We study the progressive reinforcement of preexisting social biases in Large Language Models (LLMs) Our findings reveal a progressively increasing right-leaning bias. A mechanistic interpretation identifies distinct sets of neurons responsible for model collapse and bias amplification, suggesting they arise from different underlying mechanisms.
• Score: 12.376194654498383
• License:
• Abstract: Model collapse, a phenomenon where models degrade in performance due to indiscriminate use of synthetic data is well studied. However, its role in bias amplification, the progressive reinforcement of preexisting social biases in Large Language Models (LLMs) remains underexplored. In this paper, we formally define the conditions for bias amplification and demonstrate through statistical simulations that bias can intensify even in the absence of sampling errors, the primary driver of model collapse. Empirically, we investigate political bias amplification in GPT2 using a custom built benchmark for sentence continuation tasks. Our findings reveal a progressively increasing right-leaning bias. Furthermore, we evaluate three mitigation strategies, Overfitting, Preservation, and Accumulation, and show that bias amplification persists even when model collapse is mitigated. Finally, a mechanistic interpretation identifies distinct sets of neurons responsible for model collapse and bias amplification, suggesting they arise from different underlying mechanisms.

Related papers

• How far can bias go? -- Tracing bias from pretraining data to alignment [54.51310112013655]
  This study examines the correlation between gender-occupation bias in pre-training data and their manifestation in LLMs. Our findings reveal that biases present in pre-training data are amplified in model outputs.
  arXiv  Detail & Related papers  (2024-11-28T16:20:25Z)
• An Effective Theory of Bias Amplification [18.648588509429167]
  Machine learning models may capture and amplify biases present in data, leading to disparate test performance across social groups. We propose a precise analytical theory in the context of ridge regression, where the former models neural networks in a simplified regime. Our theory offers a unified and rigorous explanation of machine learning bias, providing insights into phenomena such as bias amplification and minority-group bias.
  arXiv  Detail & Related papers  (2024-10-07T08:43:22Z)
• Editable Fairness: Fine-Grained Bias Mitigation in Language Models [52.66450426729818]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases. FAST surpasses state-of-the-art baselines with superior debiasing performance. This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Model Collapse Demystified: The Case of Regression [12.115359951879462]
  We study the phenomenon of "model collapse" in the era of proliferation of large language and image generation models. We obtain analytic formulae which quantitatively outline this phenomenon in a broad range of regimes. We propose a simple strategy based on adaptive regularization to mitigate model collapse.
  arXiv  Detail & Related papers  (2024-02-12T15:26:01Z)
• Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures [93.17009514112702]
  Pruning, setting a significant subset of the parameters of a neural network to zero, is one of the most popular methods of model compression. Despite existing evidence for this phenomenon, the relationship between neural network pruning and induced bias is not well-understood.
  arXiv  Detail & Related papers  (2023-04-25T07:42:06Z)
• A Systematic Study of Bias Amplification [16.245943270343343]
  Recent research suggests that predictions made by machine-learning models can amplify biases present in the training data. We perform the first systematic, controlled study into when and how bias amplification occurs.
  arXiv  Detail & Related papers  (2022-01-27T18:04:24Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures [62.562760228942054]
  Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective. We propose to augment the input sentences in the training data with their corresponding predicate-argument structures. We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases.
  arXiv  Detail & Related papers  (2020-10-23T16:22:05Z)
• Mitigating Gender Bias Amplification in Distribution by Posterior Regularization [75.3529537096899]
  We investigate the gender bias amplification issue from the distribution perspective. We propose a bias mitigation approach based on posterior regularization. Our study sheds the light on understanding the bias amplification.
  arXiv  Detail & Related papers  (2020-05-13T11:07:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.