On Hate Scaling Laws For Data-Swamps

• URL: http://arxiv.org/abs/2306.13141v2
• Date: Wed, 28 Jun 2023 18:00:50 GMT
• Title: On Hate Scaling Laws For Data-Swamps
• Authors: Abeba Birhane, Vinay Prabhu, Sang Han, Vishnu Naresh Boddeti
• Abstract summary: We show that the presence of hateful content in datasets, when measured with a Hate Content Rate (HCR) metric, increased by nearly $12%$. As scale increased, the tendency of the model to associate images of human faces with the human being' class over 7 other offensive classes reduced by half. For the Black female category, the tendency of the model to associate their faces with the criminal' class doubled, while quintupling for Black male faces.
• Score: 14.891493485229251
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: `Scale the model, scale the data, scale the GPU-farms' is the reigning sentiment in the world of generative AI today. While model scaling has been extensively studied, data scaling and its downstream impacts remain under explored. This is especially of critical importance in the context of visio-linguistic datasets whose main source is the World Wide Web, condensed and packaged as the CommonCrawl dump. This large scale data-dump, which is known to have numerous drawbacks, is repeatedly mined and serves as the data-motherlode for large generative models. In this paper, we: 1) investigate the effect of scaling datasets on hateful content through a comparative audit of the LAION-400M and LAION-2B-en, containing 400 million and 2 billion samples respectively, and 2) evaluate the downstream impact of scale on visio-linguistic models trained on these dataset variants by measuring racial bias of the models trained on them using the Chicago Face Dataset (CFD) as a probe. Our results show that 1) the presence of hateful content in datasets, when measured with a Hate Content Rate (HCR) metric on the inferences of the Pysentimiento hate-detection Natural Language Processing (NLP) model, increased by nearly $12\%$ and 2) societal biases and negative stereotypes were also exacerbated with scale on the models we evaluated. As scale increased, the tendency of the model to associate images of human faces with the `human being' class over 7 other offensive classes reduced by half. Furthermore, for the Black female category, the tendency of the model to associate their faces with the `criminal' class doubled, while quintupling for Black male faces. We present a qualitative and historical analysis of the model audit results, reflect on our findings and its implications for dataset curation practice, and close with a summary of our findings and potential future work to be done in this area.

Related papers

• Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERT [4.807994469764776]
  We study the influence of model scale and pre-training data on a language model's learnt social biases. Our experiments show that pre-training data substantially influences how upstream biases evolve with model scale. We shed light on the complex interplay of data and model scale, and investigate how it translates to concrete biases.
  arXiv  Detail & Related papers  (2024-07-25T23:09:33Z)
• The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models [1.6076959385522371]
  We evaluate the downstream impact of dataset scaling on visio-linguistic models trained on the LAION400-M and LAION-2B datasets. Our results show that as the training data increased, the probability of a pre-trained CLIP model misclassifying human images increased. For the smaller base ViT-B models, the probability of predicting an image of a Black man and a Latino man as criminal decreases by 20% and 47%, respectively, when the dataset is scaled from 400M to 2B samples.
  arXiv  Detail & Related papers  (2024-05-07T19:11:10Z)
• Into the LAIONs Den: Investigating Hate in Multimodal Datasets [67.21783778038645]
  This paper investigates the effect of scaling datasets on hateful content through a comparative audit of two datasets: LAION-400M and LAION-2B. We found that hate content increased by nearly 12% with dataset scale, measured both qualitatively and quantitatively. We also found that filtering dataset contents based on Not Safe For Work (NSFW) values calculated based on images alone does not exclude all the harmful content in alt-text.
  arXiv  Detail & Related papers  (2023-11-06T19:00:05Z)
• Scaling Laws Do Not Scale [54.72120385955072]
  Recent work has argued that as the size of a dataset increases, the performance of a model trained on that dataset will increase. We argue that this scaling law relationship depends on metrics used to measure performance that may not correspond with how different groups of people perceive the quality of models' output. Different communities may also have values in tension with each other, leading to difficult, potentially irreconcilable choices about metrics used for model evaluations.
  arXiv  Detail & Related papers  (2023-07-05T15:32:21Z)
• Inverse Scaling: When Bigger Isn't Better [80.42834197416444]
  Large language models (LMs) show predictable improvements to overall loss with increased scale. We present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale.
  arXiv  Detail & Related papers  (2023-06-15T20:11:23Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• Inverse scaling can become U-shaped [126.64521446943155]
  Scaling up language models has been empirically shown to improve performance on a wide range of downstream tasks. This paper takes a closer look at these inverse scaling tasks. We evaluate models of up to 540B parameters, trained on five times more compute than those evaluated in the Inverse Scaling Prize.
  arXiv  Detail & Related papers  (2022-11-03T17:26:44Z)
• Contemplating real-world object classification [53.10151901863263]
  We reanalyze the ObjectNet dataset recently proposed by Barbu et al. containing objects in daily life situations. We find that applying deep models to the isolated objects, rather than the entire scene as is done in the original paper, results in around 20-30% performance improvement.
  arXiv  Detail & Related papers  (2021-03-08T23:29:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.