Shattering the Echo Chamber: Hidden Safeguards in Manuscripts Against the AI Takeover of Peer Review

• URL: http://arxiv.org/abs/2605.05271v1
• Date: Wed, 06 May 2026 13:23:40 GMT
• Title: Shattering the Echo Chamber: Hidden Safeguards in Manuscripts Against the AI Takeover of Peer Review
• Abstract summary: End-to-End Review Outsourcing is an emerging threat.<n>We propose IntraGuard, a venue-agnostic defense framework.<n>We show that IntraGuard achieves a defense success rate of up to 84%.
• Score: 41.467123148467415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As LLMs become increasingly capable, editorial boards and program committees are growing concerned about reviewers who fully outsource peer review to commercial chatbots. This concern stems from prior findings that current chatbots lack the independent critical thinking and depth of reasoning required to assess scientific novelty. One promising direction for mitigating this concern is to embed hidden instructions into manuscripts that disrupt or alter chatbot-generated reviews. However, existing methods remain intuitive and fragile, as they typically rely on homogeneous payloads injected in an inter-stream manner, rendering them susceptible to sanitization or neutralization. In this paper, we identify End-to-End Review Outsourcing as an emerging threat and propose IntraGuard, a black-box, venue-agnostic defense framework grounded in the structural--visual decoupling inherent to the PDF. Designed for committee-side deployment, IntraGuard supports both explicit strategies that trigger refusal or warning signals, and implicit strategies that embed predefined textual markers into the generated review. These strategies can be deployed via any of three intra-stream injection mechanisms, each of which seamlessly embeds heterogeneous defensive text objects within the PDF's underlying structure without altering its visual presentation. Extensive evaluations across 7 real-world commercial chatbot settings and 12 venues spanning diverse disciplines show that IntraGuard achieves a defense success rate of up to 84%, while preserving peer-review invariance for human reviewers. IntraGuard is lightweight and hardware-independent, incurring an average overhead of only one second per manuscript on a commodity personal computer. We further evaluate 11 adaptive attacks spanning manuscript sanitization and instruction interference, and discuss the implications of constructing ensemble defenses.

Related papers

• Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle [65.6642776933861]
  This survey examines the protective paradigm that has grown around this intervention point.<n>We find that most protections are still validated mainly against static or weakly adaptive adversaries.<n>We close by consolidating cross-stage countermeasures and open problems for robust, composable, and deployable owner-side protection.
  arXiv  Detail & Related papers  (2026-08-05T00:46:50Z)
• Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Review [48.61205783234013]
  PaperGuard is the first comprehensive benchmark designed to evaluate and defend AI-generated peer-review against domain-specific, cross-modal attacks.<n>Our framework is built on three pillars: (1) a new multimodal peer-review dataset spanning multiple scientific domains; (2) a unified suite of attacks, including black-box prompt injections and white-box perturbations; and (3) a practical defense, motivated by the long-term challenge of academic papers, that uses chunk-based embedding search to efficiently localize and mitigate harmful instructions.
  arXiv  Detail & Related papers  (2026-06-10T22:06:35Z)
• How Far Did They Go? The Persuasive Tactics of Covert LLM Agents in a Discontinued Field Experiment [12.686044001239708]
  This study analyzes a publicly released dataset from a discontinued field experiment on Reddit's r/ChangeMyView.<n>The intervention, conducted by unknown, external researchers and halted following ethical backlash, involved undisclosed AI-generated accounts engaging users in live debate.<n>After public disclosure, Reddit authorized moderators to release an archive of the AI-generated comments, creating a rare opportunity to examine how large language models operated in an identity-rich deliberative forum without disclosure.
  arXiv  Detail & Related papers  (2026-06-03T15:58:32Z)
• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)
• RTD-Guard: A Black-Box Textual Adversarial Detection Framework via Replacement Token Detection [9.898508403320438]
  This paper introduces RTD-Guard, a novel black-box framework for detecting textual adversarial examples.<n>Our key insight is that word-substitution perturbations in adversarial attacks closely resemble the "replaced tokens" that a Replaced Token Detection discriminator is pre-trained to identify.<n>The entire process requires no adversarial data, model tuning, or internal model access, and uses only two black-box queries.
  arXiv  Detail & Related papers  (2026-03-13T02:30:56Z)
• Bias Injection Attacks on RAG Databases and Sanitization Defenses [9.638140849760108]
  This paper explores attacks and defenses on vector databases in retrieval-augmented generation (RAG) systems.<n>We reveal a new and subtle threat: bias injection attacks, which insert factually correct yet semantically biased passages into the knowledge base.
  arXiv  Detail & Related papers  (2025-11-30T09:27:18Z)
• ExplainableGuard: Interpretable Adversarial Defense for Large Language Models Using Chain-of-Thought Reasoning [11.340599332479705]
  Large Language Models (LLMs) are increasingly vulnerable to adversarial attacks that can subtly manipulate their outputs.<n>This paper introduces ExplainableGuard, an interpretable adversarial defense framework leveraging the chain-of-thought (CoT) reasoning capabilities of DeepSeek-Reasoner.<n>Preliminary results on the GLUE Benchmark and IMDB Movie Reviews dataset show promising defense efficacy.
  arXiv  Detail & Related papers  (2025-11-15T06:11:07Z)
• Hidden Prompts in Manuscripts Exploit AI-Assisted Peer Review [0.0]
  18 academic manuscripts on the preprint website arXiv were found to contain hidden instructions designed to manipulate AI-assisted peer review.<n>Author responses varied: one planned to withdraw the affected paper, while another defended the practice as legitimate testing of reviewer compliance.<n>We examine the technique of prompt injection in large language models (LLMs), revealing four types of hidden prompts.
  arXiv  Detail & Related papers  (2025-07-08T17:11:13Z)
• Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges [52.96987928118327]
  We find that embedding models for retrieval, rerankers, and large language model (LLM) relevance judges are vulnerable to content injection attacks.<n>We identify two primary threats: (1) inserting unrelated or harmful content within passages that still appear deceptively "relevant", and (2) inserting entire queries or key query terms into passages to boost their perceived relevance.<n>Our study systematically examines the factors that influence an attack's success, such as the placement of injected content and the balance between relevant and non-relevant material.
  arXiv  Detail & Related papers  (2025-01-30T18:02:15Z)
• Defense of Adversarial Ranking Attack in Text Retrieval: Benchmark and Baseline via Detection [12.244543468021938]
  This paper introduces two types of detection tasks for adversarial documents. A benchmark dataset is established to facilitate the investigation of adversarial ranking defense. A comprehensive investigation of the performance of several detection baselines is conducted.
  arXiv  Detail & Related papers  (2023-07-31T16:31:24Z)
• A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks [72.7373468905418]
  We develop an open-source toolkit OpenBackdoor to foster the implementations and evaluations of textual backdoor learning. We also propose CUBE, a simple yet strong clustering-based defense baseline.
  arXiv  Detail & Related papers  (2022-06-17T02:29:23Z)
• Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning [95.60856995067083]
  This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. We propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%.
  arXiv  Detail & Related papers  (2021-06-01T07:10:54Z)
• Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News [57.9843300852526]
  We introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies.
  arXiv  Detail & Related papers  (2020-09-16T14:13:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.