Detecting AI-Generated Content in Academic Peer Reviews

• URL: http://arxiv.org/abs/2602.00319v1
• Date: Fri, 30 Jan 2026 21:15:08 GMT
• Title: Detecting AI-Generated Content in Academic Peer Reviews
• Abstract summary: This study examines the temporal emergence of AI-generated content in peer reviews by applying a detection model trained on historical reviews.<n>We observe minimal detection of AI-generated content before 2022, followed by a substantial increase through 2025.
• Score: 9.46916526949695
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The growing availability of large language models (LLMs) has raised questions about their role in academic peer review. This study examines the temporal emergence of AI-generated content in peer reviews by applying a detection model trained on historical reviews to later review cycles at International Conference on Learning Representations (ICLR) and Nature Communications (NC). We observe minimal detection of AI-generated content before 2022, followed by a substantial increase through 2025, with approximately 20% of ICLR reviews and 12% of Nature Communications reviews classified as AI-generated in 2025. The most pronounced growth of AI-generated reviews in NC occurs between the third and fourth quarter of 2024. Together, these findings provide suggestive evidence of a rapidly increasing presence of AI-assisted content in peer review and highlight the need for further study of its implications for scholarly evaluation.

Related papers

• Local AI pre-screening for human triple-blind peer review in health sciences [0.0]
  An analysis of ICLR 2026 found roughly 21% of its 75,800 peer reviews were fully AI-generated.<n>We propose a triple-blind, multi-LLM pre-screening framework for peer review.
  arXiv  Detail & Related papers  (2026-07-18T00:06:33Z)
• Rejoinder: The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review [49.43514488610211]
  This article is the rejoinder to The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review''<n>We organize our response around four core themes: (i) formulating peer review as a statistical estimation problem; (ii) mitigating equity and strategic concerns in the deployment of the Isotonic Mechanism; (iii) incorporating complementary signals such as reviewer rankings and structured metadata; and (iv) exploring a human-centered framework for peer review in the era of generative AI.
  arXiv  Detail & Related papers  (2026-05-24T17:06:47Z)
• On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists [113.03797263688519]
  Many scientists view AI reviewers as probabilistic systems without the expertise to evaluate research.<n>Existing evaluations of AI reviewers have focused on whether their verdicts match human verdicts.
  arXiv  Detail & Related papers  (2026-05-20T03:33:55Z)
• AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot [45.90272517740283]
  We report the first large-scale field deployment of AI-assisted peer review.<n>Every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system.
  arXiv  Detail & Related papers  (2026-04-15T14:51:07Z)
• The Cognitive Divergence: AI Context Windows, Human Attention Decline, and the Delegation Feedback Loop [0.0]
  AI context windows have grown from 512 tokens in 2017 to 2 tokens by 2026.<n>Human Effective Context Span has declined from 16,000 tokens to 1,800.<n>The AI-to-human ratio grew from near parity at the ChatGPT launch to 556--1,111x raw and 56-111x quality-adjusted, after accounting for retrieval degradation.
  arXiv  Detail & Related papers  (2026-03-17T18:53:45Z)
• CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection [60.52240468810558]
  We introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews.<n>We also develop CoCoDet, an AI review detector via a multi-task learning framework, to achieve more accurate and robust detection of AI involvement in review content.
  arXiv  Detail & Related papers  (2025-08-28T06:03:11Z)
• Artificial Intelligence and Journalism: A Systematic Bibliometric and Thematic Analysis of Global Research [0.51795041186793]
  This study presents a comprehensive systematic review of published articles on AI in journalism from 2010 to 2025.<n>The findings show a sharp increase in research activity after 2020, with prominent focus areas including automation, misinformation, and ethical governance.<n>The review also highlights regional disparities in scholarly contributions, with limited representation from the Global South.
  arXiv  Detail & Related papers  (2025-07-15T01:11:39Z)
• The AI Imperative: Scaling High-Quality Peer Review in Machine Learning [49.87236114682497]
  We argue that AI-assisted peer review must become an urgent research and infrastructure priority.<n>We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making.
  arXiv  Detail & Related papers  (2025-06-09T18:37:14Z)
• What Makes a Good Natural Language Prompt? [72.3282960118995]
  We conduct a meta-analysis surveying more than 150 prompting-related papers from leading NLP and AI conferences from 2022 to 2025.<n>We propose a property- and human-centric framework for evaluating prompt quality, encompassing 21 properties categorized into six dimensions.<n>We then empirically explore multi-property prompt enhancements in reasoning tasks, observing that single-property enhancements often have the greatest impact.
  arXiv  Detail & Related papers  (2025-06-07T23:19:27Z)
• Neuro-Symbolic AI in 2024: A Systematic Review [0.29260385019352086]
  The review followed the PRISMA methodology, utilizing databases such as IEEE Explore, Google Scholar, arXiv, ACM, and SpringerLink.<n>From an initial pool of 1,428 papers, 167 met the inclusion criteria and were analyzed in detail.<n>The majority of research efforts are concentrated in the areas of learning and inference, logic and reasoning, and knowledge representation.
  arXiv  Detail & Related papers  (2025-01-09T18:48:35Z)
• Publication Trends in Artificial Intelligence Conferences: The Rise of Super Prolific Authors [1.5998912722142724]
  We analyzed 87,137 papers from 11 AI conferences to examine publication trends over the past decade.<n>Our findings reveal a consistent increase in both the number of papers and authors, reflecting the growing interest in AI research.<n>In light of this analysis, the AI research community should consider revisiting authorship policies, addressing equity concerns, and evaluating the workload of junior researchers.
  arXiv  Detail & Related papers  (2024-11-28T06:56:49Z)
• AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment [54.93996119324928]
  We create the largest AIGI subjective quality database to date with 20,000 AIGIs and 420,000 subjective ratings, known as AIGIQA-20K. We conduct benchmark experiments on this database to assess the correspondence between 16 mainstream AIGI quality models and human perception.
  arXiv  Detail & Related papers  (2024-04-04T12:12:24Z)
• Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews [51.453135368388686]
  We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM) Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level.
  arXiv  Detail & Related papers  (2024-03-11T21:51:39Z)
• Approaches to Generative Artificial Intelligence, A Social Justice Perspective [0.0]
  Rise of AI-driven writing assistance, dubbed 'AI-giarism' by Chan, will make plagiarism more accessible and less detectable. This paper aims to explore generative AI from a social justice perspective, examining the training of these models, the inherent biases, and the potential injustices in detecting AI-generated writing.
  arXiv  Detail & Related papers  (2023-08-17T06:30:46Z)
• Characterising Research Areas in the field of AI [68.8204255655161]
  We identified the main conceptual themes by performing clustering analysis on the co-occurrence network of topics. The results highlight the growing academic interest in research themes like deep learning, machine learning, and internet of things.
  arXiv  Detail & Related papers  (2022-05-26T16:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.