Most biomedical publications show signs of LLM-assisted writing

• URL: http://arxiv.org/abs/2608.10715v1
• Date: Tue, 11 Aug 2026 09:34:18 GMT
• Title: Most biomedical publications show signs of LLM-assisted writing
• Abstract summary: We suggest and validate a new unbiased approach to estimate LLM usage in a corpus of texts based on changing word frequencies.<n>We apply our method to the full texts of open-access biomedical papers from Pubmed Central.<n>We find that LLMs are twice as likely to be used when writing a paragraph in the Discussion section (68%) compared to a paragraph in the Methods section (32%), but even inside the Methods section, the overall prevalence of LLM usage is over 50%.
• Score: 5.836991649815996
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Over the past several years, LLM-powered chatbots and agents have become widely used as a tool for academic writing. LLM-assisted writing can be valuable by removing language barriers but at the same time causes concerns about misconduct and fraud. To inform policy decisions, it is necessary to monitor the prevalence of LLM-altered texts in scholarly publications. Despite some recent progress in this direction, no existing method can produce reliable estimates. Here we suggest and validate a new unbiased approach to estimate LLM usage in a corpus of texts based on changing word frequencies. We apply our method to the full texts of open-access biomedical papers from Pubmed Central, and show that by the end of 2025, 89% of papers show excess of LLM-associated vocabulary. We also find that LLMs are twice as likely to be used when writing a paragraph in the Discussion section (68%) compared to a paragraph in the Methods section (32%), but even inside the Methods section, the overall prevalence of LLM usage is over 50%. We believe that our estimates are crucial to shape future guidelines and policies.

Related papers

• How LLMs Distort Our Written Language [11.98582470524476]
  Large language models (LLMs) are used by over a billion people globally, most often to assist with writing.<n>We demonstrate that LLMs not only alter the voice and tone of human writing, but also consistently alter the intended meaning.
  arXiv  Detail & Related papers  (2026-03-18T18:01:52Z)
• Counterfactual LLM-based Framework for Measuring Rhetorical Style [15.917819866091191]
  We introduce a counterfactual, LLM-based framework to disentangle rhetorical style from substantive content in machine learning papers.<n>Applying this method to 8,485 ICLR submissions sampled from 2017 to 2025, we generate more than 250,000 counterfactual writings.<n>We find that visionary framing significantly predicts downstream attention, including citations and media attention, even after controlling for peer-review evaluations.
  arXiv  Detail & Related papers  (2025-12-22T22:22:46Z)
• Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review [66.73247554182376]
  Large language models (LLMs) have led to their integration into peer review.<n>The unchecked adoption of LLMs poses significant risks to the integrity of the peer review system.<n>We show that manipulating 5% of the reviews could potentially cause 12% of the papers to lose their position in the top 30% rankings.
  arXiv  Detail & Related papers  (2024-12-02T16:55:03Z)
• Delving into LLM-assisted writing in biomedical publications through excess vocabulary [4.58733012283457]
  Large language models (LLMs) like ChatGPT can generate and revise text with human-level performance.<n>We study vocabulary changes in over 15 million biomedical abstracts from 2010--2024 indexed by PubMed.<n>We show that LLMs have had an unprecedented impact on scientific writing in biomedical research, surpassing the effect of major world events such as the Covid pandemic.
  arXiv  Detail & Related papers  (2024-06-11T07:16:34Z)
• ReMoDetect: Reward Models Recognize Aligned LLM's Generations [55.06804460642062]
  Large language models (LLMs) generate human-preferable texts. In this paper, we identify the common characteristics shared by these models. We propose two training schemes to further improve the detection ability of the reward model.
  arXiv  Detail & Related papers  (2024-05-27T17:38:33Z)
• Mapping the Increasing Use of LLMs in Scientific Papers [99.67983375899719]
  We conduct the first systematic, large-scale analysis across 950,965 papers published between January 2020 and February 2024 on the arXiv, bioRxiv, and Nature portfolio journals. Our findings reveal a steady increase in LLM usage, with the largest and fastest growth observed in Computer Science papers.
  arXiv  Detail & Related papers  (2024-04-01T17:45:15Z)
• Large Language Models: A Survey [66.39828929831017]
  Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks.<n>LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
  arXiv  Detail & Related papers  (2024-02-09T05:37:09Z)
• LM-Polygraph: Uncertainty Estimation for Language Models [71.21409522341482]
  Uncertainty estimation (UE) methods are one path to safer, more responsible, and more effective use of large language models (LLMs) We introduce LM-Polygraph, a framework with implementations of a battery of state-of-the-art UE methods for LLMs in text generation tasks, with unified program interfaces in Python. It introduces an extendable benchmark for consistent evaluation of UE techniques by researchers, and a demo web application that enriches the standard chat dialog with confidence scores.
  arXiv  Detail & Related papers  (2023-11-13T15:08:59Z)
• Prevalence and prevention of large language model use in crowd work [11.554258761785512]
  We show that the use of large language models (LLMs) is prevalent among crowd workers. We show that targeted mitigation strategies can significantly reduce, but not eliminate, LLM use.
  arXiv  Detail & Related papers  (2023-10-24T09:52:09Z)
• Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models [17.993892458845124]
  We propose a black-box method to predict document-level membership and instantiate it on OpenLLaMA-7B. We show our approach to outperform the sentence-level membership inference attacks used in the privacy literature for the document-level membership task.
  arXiv  Detail & Related papers  (2023-10-23T15:00:46Z)
• Red Teaming Language Model Detectors with Language Models [114.36392560711022]
  Large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. Recent works have proposed algorithms to detect LLM-generated text and protect LLMs. We study two types of attack strategies: 1) replacing certain words in an LLM's output with their synonyms given the context; 2) automatically searching for an instructional prompt to alter the writing style of the generation.
  arXiv  Detail & Related papers  (2023-05-31T10:08:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.