How LLMs Distort Our Written Language

• URL: http://arxiv.org/abs/2603.18161v1
• Date: Wed, 18 Mar 2026 18:01:52 GMT
• Title: How LLMs Distort Our Written Language
• Abstract summary: Large language models (LLMs) are used by over a billion people globally, most often to assist with writing.<n>We demonstrate that LLMs not only alter the voice and tone of human writing, but also consistently alter the intended meaning.
• Score: 11.98582470524476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are used by over a billion people globally, most often to assist with writing. In this work, we demonstrate that LLMs not only alter the voice and tone of human writing, but also consistently alter the intended meaning. First, we conduct a human user study to understand how people actually interact with LLMs when using them for writing. Our findings reveal that extensive LLM use led to a nearly 70% increase in essays that remained neutral in answering the topic question. Significantly more heavy LLM users reported that the writing was less creative and not in their voice. Next, using a dataset of human-written essays that was collected in 2021 before the widespread release of LLMs, we study how asking an LLM to revise the essay based on the human-written feedback in the dataset induces large changes in the resulting content and meaning. We find that even when LLMs are prompted with expert feedback and asked to only make grammar edits, they still change the text in a way that significantly alters its semantic meaning. We then examine LLM-generated text in the wild, specifically focusing on the 21% of AI-generated scientific peer reviews at a recent top AI conference. We find that LLM-generated reviews place significantly less weight on clarity and significance of the research, and assign scores that, on average, are a full point higher.These findings highlight a misalignment between the perceived benefit of AI use and an implicit, consistent effect on the semantics of human writing, motivating future work on how widespread AI writing will affect our cultural and scientific institutions.

Related papers

• Most biomedical publications show signs of LLM-assisted writing [5.836991649815996]
  We suggest and validate a new unbiased approach to estimate LLM usage in a corpus of texts based on changing word frequencies.<n>We apply our method to the full texts of open-access biomedical papers from Pubmed Central.<n>We find that LLMs are twice as likely to be used when writing a paragraph in the Discussion section (68%) compared to a paragraph in the Methods section (32%), but even inside the Methods section, the overall prevalence of LLM usage is over 50%.
  arXiv  Detail & Related papers  (2026-08-11T09:34:18Z)
• Data, Not Model: Explaining Bias toward LLM Texts in Neural Retrievers [76.92519309816008]
  Recent studies show that neural retrievers often display source bias, favoring passages generated by LLMs over human-written ones.<n>This bias has been considered an inherent flaw of retrievers, raising concerns about the fairness and reliability of modern information access systems.<n>Our work challenges this view by showing that source bias stems from supervision in retrieval datasets rather than the models themselves.
  arXiv  Detail & Related papers  (2026-04-07T17:57:07Z)
• LLM or Human? Perceptions of Trust and Information Quality in Research Summaries [4.14522715241162]
  Large Language Models (LLMs) are increasingly used to generate and edit scientific abstracts.<n>Despite growing adoption, little is known about how readers perceive LLM-generated summaries.
  arXiv  Detail & Related papers  (2026-01-22T00:53:38Z)
• Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages [51.96666324242191]
  We analyze whether user utilization of novel writing assistants in a charity advertisement writing task is affected by the AI's performance in a second language.<n>We quantify the extent to which these patterns translate into the persuasiveness of generated charity advertisements.
  arXiv  Detail & Related papers  (2025-02-13T17:49:30Z)
• IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance [49.605835017477034]
  IssueBench is a set of 2.49m realistic English-language prompts to measure issue bias in large language models.<n>Using IssueBench, we show that issue biases are common and persistent in 10 state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2025-02-12T13:37:03Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits [39.00434175773803]
  We hired professional writers to edit paragraphs in several creative domains.<n>We curated the LAMP corpus: 1,057 LLM-generated paragraphs edited by professional writers according to our taxonomy.<n>Analysis of LAMP reveals that none of the LLMs used in our study outperform each other in terms of writing quality.
  arXiv  Detail & Related papers  (2024-09-22T16:13:00Z)
• LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing [106.45895712717612]
  Large language models (LLMs) have shown remarkable versatility in various generative tasks. This study focuses on the topic of LLMs assist NLP Researchers. To our knowledge, this is the first work to provide such a comprehensive analysis.
  arXiv  Detail & Related papers  (2024-06-24T01:30:22Z)
• Delving into LLM-assisted writing in biomedical publications through excess vocabulary [4.58733012283457]
  Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance.<n>We study vocabulary changes in over 15 million biomedical abstracts from 2010--2024 indexed by PubMed.<n>We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the Covid pandemic.
  arXiv  Detail & Related papers  (2024-06-11T07:16:34Z)
• Large Language Models: A Survey [66.39828929831017]
  Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks.<n>LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
  arXiv  Detail & Related papers  (2024-02-09T05:37:09Z)
• Do LLMs exhibit human-like response biases? A case study in survey design [66.1850490474361]
  We investigate the extent to which large language models (LLMs) reflect human response biases, if at all. We design a dataset and framework to evaluate whether LLMs exhibit human-like response biases in survey questionnaires. Our comprehensive evaluation of nine models shows that popular open and commercial LLMs generally fail to reflect human-like behavior.
  arXiv  Detail & Related papers  (2023-11-07T15:40:43Z)
• Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong [35.64962031447787]
  Large Language Models (LLMs) are increasingly used for accessing information on the web. Our experiments with 80 crowdworkers compare language models with search engines (information retrieval systems) at facilitating fact-checking. Users reading LLM explanations are significantly more efficient than those using search engines while achieving similar accuracy.
  arXiv  Detail & Related papers  (2023-10-19T08:09:58Z)
• Event knowledge in large language models: the gap between the impossible and the unlikely [46.540380831486125]
  We show that pre-trained large language models (LLMs) possess substantial event knowledge. They almost always assign higher likelihood to possible vs. impossible events. However, they show less consistent preferences for likely vs. unlikely events.
  arXiv  Detail & Related papers  (2022-12-02T23:43:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.