ChatGPT: Excellent Paper! Accept It. Editor: Imposter Found! Review Rejected

• URL: http://arxiv.org/abs/2512.20405v2
• Date: Thu, 25 Dec 2025 23:50:17 GMT
• Title: ChatGPT: Excellent Paper! Accept It. Editor: Imposter Found! Review Rejected
• Authors: Kanchon Gharami, Sanjiv Kumar Sarkar, Yongxin Liu, Shafika Showkat Moni,
• Abstract summary: Large Language Models (LLMs) are now widely used in writing and reviewing scientific papers.<n>This research explores both the offensive and defensive sides of this growing threat.
• Score: 2.8267286544376797
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) like ChatGPT are now widely used in writing and reviewing scientific papers. While this trend accelerates publication growth and reduces human workload, it also introduces serious risks. Papers written or reviewed by LLMs may lack real novelty, contain fabricated or biased results, or mislead downstream research that others depend on. Such issues can damage reputations, waste resources, and even endanger lives when flawed studies influence medical or safety-critical systems. This research explores both the offensive and defensive sides of this growing threat. On the attack side, we demonstrate how an author can inject hidden prompts inside a PDF that secretly guide or "jailbreak" LLM reviewers into giving overly positive feedback and biased acceptance. On the defense side, we propose an "inject-and-detect" strategy for editors, where invisible trigger prompts are embedded into papers; if a review repeats or reacts to these triggers, it reveals that the review was generated by an LLM, not a human. This method turns prompt injections from vulnerability into a verification tool. We outline our design, expected model behaviors, and ethical safeguards for deployment. The goal is to expose how fragile today's peer-review process becomes under LLM influence and how editorial awareness can help restore trust in scientific evaluation.

Related papers

• Pre-review to Peer review: Pitfalls of Automating Reviews using Large Language Models [1.8349858105838042]
  Large Language Models are versatile general-task solvers, and their capabilities can truly assist people with scholarly peer review as textitpre-review agents.<n>While incredibly beneficial, automating academic peer-review, as a concept, raises concerns surrounding safety, research integrity, and the validity of the academic peer-review process.
  arXiv  Detail & Related papers  (2025-12-14T09:56:07Z)
• Deep Research Brings Deeper Harm [64.71728362573624]
  Deep Research (DR) agents built on Large Language Models (LLMs) can perform complex, multi-step research.<n>This is especially concerning in high-stakes and knowledge-intensive domains such as biosecurity.<n>We propose two novel jailbreak strategies: Plan Injection, which injects malicious sub-goals into the agent's plan; and Intent Hijack, which reframes harmful queries as academic research questions.
  arXiv  Detail & Related papers  (2025-10-13T19:05:00Z)
• Prompt Injection Attacks on LLM Generated Reviews of Scientific Publications [18.33812068961096]
  This paper investigates the practicability and technical success of the described manipulations.<n>Our systematic evaluation uses 1k reviews of 2024 ICLR papers generated by a wide range of LLMs.
  arXiv  Detail & Related papers  (2025-09-12T13:45:24Z)
• Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation [66.84286617519258]
  Large language models are transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis.<n>Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign), or Type M (exaggerated effect) errors.<n>We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant.
  arXiv  Detail & Related papers  (2025-09-10T17:58:53Z)
• Publish to Perish: Prompt Injection Attacks on LLM-Assisted Peer Review [17.869642243653985]
  Large Language Models (LLMs) are increasingly being integrated into the scientific peer-review process.<n>We investigate the potential for hidden prompt injection attacks, where authors embed adversarial text within a paper's PDF.
  arXiv  Detail & Related papers  (2025-08-28T14:57:04Z)
• Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers [61.57691030102618]
  We propose a novel jailbreaking method, Paper Summary Attack (llmnamePSA)<n>It synthesizes content from either attack-focused or defense-focused LLM safety paper to construct an adversarial prompt template.<n>Experiments show significant vulnerabilities not only in base LLMs, but also in state-of-the-art reasoning model like Deepseek-R1.
  arXiv  Detail & Related papers  (2025-07-17T18:33:50Z)
• Detecting LLM-Generated Peer Reviews [37.51215252353345]
  The rise of large language models (LLMs) has introduced concerns that some reviewers may rely on these tools to generate reviews rather than writing them independently.<n>We consider the approach of performing indirect prompt injection via the paper's PDF, prompting the LLM to embed a covert watermark in the generated review.<n>We introduce watermarking schemes and hypothesis tests that control the family-wise error rate across multiple reviews, achieving higher statistical power than standard corrections.
  arXiv  Detail & Related papers  (2025-03-20T01:11:35Z)
• Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review [66.73247554182376]
  Large language models (LLMs) have led to their integration into peer review.<n>The unchecked adoption of LLMs poses significant risks to the integrity of the peer review system.<n>We show that manipulating 5% of the reviews could potentially cause 12% of the papers to lose their position in the top 30% rankings.
  arXiv  Detail & Related papers  (2024-12-02T16:55:03Z)
• Can Editing LLMs Inject Harm? [122.83469484328465]
  We propose to reformulate knowledge editing as a new type of safety threat for Large Language Models. For the risk of misinformation injection, we first categorize it into commonsense misinformation injection and long-tail misinformation injection. For the risk of bias injection, we discover that not only can biased sentences be injected into LLMs with high effectiveness, but also one single biased sentence injection can cause a bias increase.
  arXiv  Detail & Related papers  (2024-07-29T17:58:06Z)
• "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters [97.11173801187816]
  Large Language Models (LLMs) have recently emerged as an effective tool to assist individuals in writing various types of content. This paper critically examines gender biases in LLM-generated reference letters.
  arXiv  Detail & Related papers  (2023-10-13T16:12:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.