Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community
- URL: http://arxiv.org/abs/2606.10159v1
- Date: Mon, 08 Jun 2026 20:38:06 GMT
- Title: Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community
- Abstract summary: We show that AI-mediated peer review is vulnerable to a simple, low-cost manipulation.<n>We see this across disciplines and publication venues, for both human-written and AI-generated papers.<n>Our results suggest that AI tools should not be treated as neutral evaluators in high-stakes peer review.
- Score: 33.39468805524881
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: AI is increasingly used to support scientific peer review, from manuscript screening, reviewer assistance to editorial triage. Although such systems promise to reduce reviewer burden and accelerate publication, their robustness to strategic manipulation remains poorly understood. Here we show that AI-mediated peer review is vulnerable to a simple, low-cost manipulation: superficial rephrasing of the manuscript abstract. Without changing the underlying scientific content and communication, and even without knowledge of the reviewing model, adversarially rewritten abstracts substantially improve AI review outcomes. We see this across disciplines and publication venues, for both human-written and AI-generated papers. Our strongest attack achieves an attack-success-rate of about 38%, increasing acceptance ratings by +1.31 for Gemini 3 Flash reviewers and by +0.88 for GPT 5.4 Mini reviewers on a 10-point scale. When the original AI review suggests 'reject', the success rate rises to more than 50%. This effect extends beyond overall score inflation, increasing review confidence and scores on core scientific criteria such as soundness, significance and perceived contribution. The attack is practical, requiring only about 5 minutes and $1 for a 10-page AI conference submission, and is hard to distinguish from ordinary scientific editing. Inflated AI reviews could bias downstream human decision-making, shifting editorial recommendations from rejection towards acceptance. These findings reveal a general vulnerability in AI-assisted scientific evaluation: when AI-generated review influence editorial decisions, authors may be incentivized to optimize manuscripts for AI judgment rather than scientific merit. Our results suggest that AI tools should not be treated as neutral evaluators in high-stakes peer review without systematic robustness testing, transparent safeguards and careful human oversight.
Related papers
- No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions [43.03725702558361]
We study a harder and more policy-relevant failure mode: no hidden text, no prompt injection, and no changes to methods, experiments, figures, equations, proofs, or numerical results.<n>The attacker modifies only presentation-level content, such as the abstract, contribution framing, related work, discussion, and narrative structure.<n>Across three mainstream AI reviewers, adversarial repackaging achieves a 75.1% attack success rate and a mean score gain of +1.21/10.
arXiv Detail & Related papers (2026-06-11T08:30:18Z) - Can AI Review Improve Paper Drafting? An Empirical Study on 20 Computer Architecture Submissions [4.04252733799147]
We study 20 computer architecture papers, with varying levels of submission lineage, to expose how well AI review aligns with human review.<n>To conduct the case study, we build a web UI-integrated tool, emphAI-Paper-Review, that generates structured AI review of a draft paper.
arXiv Detail & Related papers (2026-05-31T05:05:26Z) - On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists [113.03797263688519]
Many scientists view AI reviewers as probabilistic systems without the expertise to evaluate research.<n>Existing evaluations of AI reviewers have focused on whether their verdicts match human verdicts.
arXiv Detail & Related papers (2026-05-20T03:33:55Z) - Stop Automating Peer Review Without Rigorous Evaluation [51.53099943385505]
This position paper argues that today's AI systems should not be used to produce paper reviews.<n>We identify two critical issues: 1) AI reviewers exhibit a hivemind effect of excessive agreement within and across papers that reduces perspective diversity.<n>We argue that addressing the peer review crisis requires a science of peer review automation.
arXiv Detail & Related papers (2026-05-04T22:41:04Z) - When AI reviews science: Can we trust the referee? [73.47745294608072]
We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
arXiv Detail & Related papers (2026-04-26T08:03:32Z) - Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews [69.66583722746904]
We introduce Beyond Rating, a holistic evaluation framework that assesses AI reviewers across five dimensions.<n>We propose a Max-Recall strategy to accommodate valid expert disagreement and introduce a curated dataset of paper with high-confidence reviews.<n>Our proposed text-centric metrics--particularly the recall of weakness arguments--correlate strongly with rating accuracy.
arXiv Detail & Related papers (2026-04-21T14:21:15Z) - AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot [45.90272517740283]
We report the first large-scale field deployment of AI-assisted peer review.<n>Every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system.
arXiv Detail & Related papers (2026-04-15T14:51:07Z) - The Great AI Witch Hunt: Reviewers Perception and (Mis)Conception of Generative AI in Research Writing [36.188062803005515]
Generative AI (GenAI) use in research writing is growing fast.<n>It is unclear how peer reviewers recognize or misjudge AI-augmented manuscripts.<n>Our findings indicate that while AI-augmented writing improves readability, language diversity, and informativeness, it often lacks research details and reflective insights from authors.
arXiv Detail & Related papers (2024-06-27T02:38:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.