AI AI Bias: Large Language Models Favor Their Own Generated Content

• URL: http://arxiv.org/abs/2407.12856v1
• Date: Tue, 9 Jul 2024 13:15:14 GMT
• Title: AI AI Bias: Large Language Models Favor Their Own Generated Content
• Authors: Walter Laurito, Benjamin Davis, Peli Grietzer, Tomáš Gavenčiak, Ada Böhm, Jan Kulveit,
• Abstract summary: We test whether large language models (LLMs) are biased towards text generated by LLMs over text authored by humans. Our results show a consistent tendency for LLM-based AIs to prefer LLM-generated content. This suggests the possibility of AI systems implicitly discriminating against humans, giving AI agents an unfair advantage.
• Score: 0.1979158763744267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Are large language models (LLMs) biased towards text generated by LLMs over text authored by humans, leading to possible anti-human bias? Utilizing a classical experimental design inspired by employment discrimination studies, we tested widely-used LLMs, including GPT-3.5 and GPT4, in binary-choice scenarios. These involved LLM-based agents selecting between products and academic papers described either by humans or LLMs under identical conditions. Our results show a consistent tendency for LLM-based AIs to prefer LLM-generated content. This suggests the possibility of AI systems implicitly discriminating against humans, giving AI agents an unfair advantage.

Related papers

• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective [66.34066553400108]
  We conduct a rigorous evaluation of Large Language Models' implicit bias towards certain groups by attacking them with carefully crafted instructions to elicit biased responses. We propose three attack approaches, i.e., Disguise, Deception, and Teaching, based on which we built evaluation datasets for four common bias types.
  arXiv  Detail & Related papers  (2024-06-20T06:42:08Z)
• ReMoDetect: Reward Models Recognize Aligned LLM's Generations [55.06804460642062]
  Large language models (LLMs) generate human-preferable texts. In this paper, we identify the common characteristics shared by these models. We propose two training schemes to further improve the detection ability of the reward model.
  arXiv  Detail & Related papers  (2024-05-27T17:38:33Z)
• Towards Human-Level Text Coding with LLMs: The Case of Fatherhood Roles in Public Policy Documents [19.65846717628022]
  Large language models (LLMs) promise automation with better results and less programming. In this study, we evaluate LLMs on three original coding tasks involving typical complexities encountered in political science settings. We find that the best prompting strategy consists of providing the LLMs with a detailed codebook, as the one provided to human coders.
  arXiv  Detail & Related papers  (2023-11-20T15:34:45Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• Harnessing the Power of LLMs: Evaluating Human-AI Text Co-Creation through the Lens of News Headline Generation [58.31430028519306]
  This study explores how humans can best leverage LLMs for writing and how interacting with these models affects feelings of ownership and trust in the writing process. While LLMs alone can generate satisfactory news headlines, on average, human control is needed to fix undesirable model outputs.
  arXiv  Detail & Related papers  (2023-10-16T15:11:01Z)
• Verbosity Bias in Preference Labeling by Large Language Models [10.242500241407466]
  We examine the biases that come along with evaluating Large Language Models (LLMs) We take a closer look into verbosity bias -- a bias where LLMs sometimes prefer more verbose answers even if they have similar qualities.
  arXiv  Detail & Related papers  (2023-10-16T05:19:02Z)
• Queer People are People First: Deconstructing Sexual Identity Stereotypes in Large Language Models [3.974379576408554]
  Large Language Models (LLMs) are trained primarily on minimally processed web text. LLMs can inadvertently perpetuate stereotypes towards marginalized groups, like the LGBTQIA+ community.
  arXiv  Detail & Related papers  (2023-06-30T19:39:01Z)
• Potential Benefits of Employing Large Language Models in Research in Moral Education and Development [0.0]
  Recently, computer scientists have developed large language models (LLMs) by training prediction models with large-scale language corpora and human reinforcements. I will examine how LLMs might contribute to moral education and development research.
  arXiv  Detail & Related papers  (2023-06-23T22:39:05Z)
• Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks [12.723777984461693]
  Large language models (LLMs) are remarkable data annotators. Crowdsourcing, an important, inexpensive way to obtain human annotations, may itself be impacted by LLMs. We estimate that 33-46% of crowd workers used LLMs when completing a task.
  arXiv  Detail & Related papers  (2023-06-13T16:46:24Z)
• On Learning to Summarize with Large Language Models as References [101.79795027550959]
  Large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets. We study an LLM-as-reference learning setting for smaller text summarization models to investigate whether their performance can be substantially improved.
  arXiv  Detail & Related papers  (2023-05-23T16:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.