Counter-GEO-Bench: Evaluating Defenses Against Information-Distorting Generative Engine Optimization

• URL: http://arxiv.org/abs/2609.02316v1
• Date: Wed, 02 Sep 2026 09:01:51 GMT
• Title: Counter-GEO-Bench: Evaluating Defenses Against Information-Distorting Generative Engine Optimization
• Abstract summary: Generative engine optimization (GEO) enables content producers to increase the visibility of their web pages in generative search engines.<n>The same techniques can deliver targeted misinformation when adversaries publish ordinary-looking GEO-optimized documents.<n>We present Counter-GEO-Bench, a defense benchmark that pairs 247 human-verified, quality-gated queries with information-preserving and information-distorting GEO rewrites.
• Score: 48.93628521719709
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative engine optimization (GEO) enables content producers to increase the visibility of their web pages in generative search engines, but the same techniques can deliver targeted misinformation when adversaries publish ordinary-looking GEO-optimized documents that victim large language models (LLMs) retrieve and synthesize into distorted answers. No existing benchmark evaluates defenses against this threat under controlled conditions. Therefore, we present Counter-GEO-Bench, a defense benchmark that pairs 247 human-verified, quality-gated queries with information-preserving and information-distorting GEO rewrites, and evaluates defenses on attack success rate (ASR), false positive rate, and answer quality across three victim LLMs. Under Counter-GEO-Bench, three off-the-shelf defenses (Granite Guardian, Llama Guard 3, and NeMo Self-Check Fact-Checking) reduce ASR by at most 5.7% relative, while Granite Guardian's reduction is not statistically significant. Safety-taxonomy guardrails target policy violations, while GEO misinformation passes through them as fluent informational content. To this end, a lightweight benchmark baseline, C-GEO Guard, is proposed, reducing ASR by 47.6% relative with near-zero utility loss, which proves threat tractable.

Related papers

• When Optimization Becomes Manipulation: Defending Generative Search against Malicious Generative Engine Optimization [3.9144095238039855]
  This paper focuses on defending generative search engines against malicious Generative Engine Optimization (GEO)<n>Recent GEO methods have advanced from hand-crafted rewriting to automated and agentic optimization.<n>We propose GEO Defender, a two-stage defense aligned with the attack chain that requires no fine-tuning of the target LLM.
  arXiv  Detail & Related papers  (2026-09-02T07:13:40Z)
• RAGSentinel: Certifiable Geometric Consensus for Robust Retrieval-Augmented Generation [9.458292549037155]
  We propose RAGSentinel, a training-free, label-free defense for black-box RAG systems.<n>RaGSentinel consistently achieves low attack success rates while preserving competitive accuracy.
  arXiv  Detail & Related papers  (2026-08-25T01:44:07Z)
• SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents [13.658613657290166]
  We study whether recommendation agents preserve utility-aligned decisions when seller-controlled sources are rewritten for GEO.<n>We empirically show that GEO attacks can promote flawed target products.<n>Simple defenses, including defensive prompting and structured evidence checks, reduce harmful target promotion by up to 39.2%.
  arXiv  Detail & Related papers  (2026-06-08T22:40:08Z)
• GEO-Bench: Benchmarking Ranking Manipulation in Generative Engine Optimization [9.762893142689357]
  We present GEO-Bench, a benchmark that evaluates GEO ranking-manipulation attacks under one protocol.<n>It unifies black-box prompt-based attacks (TAP, Zero-Shot), white-box gradient-based attacks (STS, RAF, StealthRank), and ten white-hat C-SEO strategies.<n>We score every method on five datasets against a fixed open-weight ranker.
  arXiv  Detail & Related papers  (2026-05-27T21:10:43Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• External Data Extraction Attacks against Retrieval-Augmented Large Language Models [70.47869786522782]
  RAG has emerged as a key paradigm for enhancing large language models (LLMs)<n>RAG introduces new risks of external data extraction attacks (EDEAs), where sensitive or copyrighted data in its knowledge base may be extracted verbatim.<n>We present the first comprehensive study to formalize EDEAs against retrieval-augmented LLMs.
  arXiv  Detail & Related papers  (2025-10-03T12:53:45Z)
• Benchmarking Misuse Mitigation Against Covert Adversaries [80.74502950627736]
  Existing language model safety evaluations focus on overt attacks and low-stakes tasks.<n>We develop Benchmarks for Stateful Defenses (BSD), a data generation pipeline that automates evaluations of covert attacks and corresponding defenses.<n>Our evaluations indicate that decomposition attacks are effective misuse enablers, and highlight stateful defenses as a countermeasure.
  arXiv  Detail & Related papers  (2025-06-06T17:33:33Z)
• The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems [101.68501850486179]
  We explore adversarial attacks against retrieval-augmented generation (RAG) systems to identify their vulnerabilities.<n>This task aims to find imperceptible perturbations that retrieve a target document, originally excluded from the initial top-$k$ candidate set.<n>We propose ReGENT, a reinforcement learning-based framework that tracks interactions between the attacker and the target RAG.
  arXiv  Detail & Related papers  (2025-05-24T08:19:25Z)
• Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-of-Service Attacks on Retrieval-Augmented Generation of LLMs [8.09404178079053]
  Retrieval-Augmented Generation (RAG) integrates Large Language Models (LLMs) with external knowledge bases, improving output quality while introducing new security risks.<n>Existing studies on RAG vulnerabilities typically focus on exploiting the retrieval mechanism to inject erroneous knowledge or malicious texts, inducing incorrect outputs.<n>In this paper, we uncover a novel vulnerability: the safety guardrails of LLMs, while designed for protection, can also be exploited as an attack vector by adversaries. Building on this vulnerability, we propose MutedRAG, a novel denial-of-service attack that reversely leverages the guardrails to undermine the availability of
  arXiv  Detail & Related papers  (2025-04-30T14:18:11Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal [64.9938658716425]
  SORRY-Bench is a proposed benchmark for evaluating large language models' (LLMs) ability to recognize and reject unsafe user requests.<n>First, existing methods often use coarse-grained taxonomy of unsafe topics, and are over-representing some fine-grained topics.<n>Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)
• Improved Generation of Adversarial Examples Against Safety-aligned LLMs [72.38072942860309]
  Adversarial prompts generated using gradient-based methods exhibit outstanding performance in performing automatic jailbreak attacks against safety-aligned LLMs. In this paper, we explore a new perspective on this problem, suggesting that it can be alleviated by leveraging innovations inspired in transfer-based attacks. We show that 87% of the query-specific adversarial suffixes generated by the developed combination can induce Llama-2-7B-Chat to produce the output that exactly matches the target string on AdvBench.
  arXiv  Detail & Related papers  (2024-05-28T06:10:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.