REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

• URL: http://arxiv.org/abs/2605.12813v1
• Date: Tue, 12 May 2026 23:13:50 GMT
• Title: REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations
• Abstract summary: We formulate hallucination elicitation as a constrained optimization problem.<n>The goal is to find semantically coherent adversarial prompts that are equivalent to benign user prompts.<n>We propose REALISTA, a realistic latent-space attack framework.
• Score: 60.451310947394596
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) achieve strong performance across many tasks but remain vulnerable to hallucinations, motivating the need for realistic adversarial prompts that elicit such failures. We formulate hallucination elicitation as a constrained optimization problem, where the goal is to find semantically coherent adversarial prompts that are equivalent to benign user prompts. Existing methods remain limited: discrete prompt-based attacks preserve semantic equivalence and coherence but search only over a limited set of prompt variations, while continuous latent-space attacks explore a richer space but often decode into prompts that are no longer valid rephrasings. To address these limitations, we propose REALISTA, a realistic latent-space attack framework. REALISTA constructs an input-dependent dictionary of valid editing directions, each corresponding to a semantically equivalent and coherent rephrasing, and optimizes continuous combinations of these directions in latent space. This design combines the optimization flexibility of continuous attacks with the semantic realism of discrete rephrasing-based attacks. Experiments demonstrate that REALISTA achieves superior or comparable performance to state-of-the-art realistic attacks on open-source LLMs and, crucially, succeeds in attacking large reasoning models under free-form response settings, where prior realistic attacks fail. Code is available at https://github.com/Buyun-Liang/REALISTA.

Related papers

• Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Prompts [14.233668486426795]
  Large language models (LLMs) excel in reasoning and knowledge-intensive tasks but remain vulnerable to prompt-level adversarial attacks.<n>We propose an A*-inspired Factual Error Induction Framework, a framework for generating semantically aligned yet obfuscated prompts.<n>We prove that prompt rewriting follows a contractive recurrence, leading to semantic collapse as $$ decreases.
  arXiv  Detail & Related papers  (2026-05-31T20:20:53Z)
• LLM-Agnostic Semantic Representation Attack [18.00668872674083]
  We propose a novel LLM-agnostic paradigm that fundamentally reconceptualizes adversarial objectives from exact textual targeting to malicious semantic representations.<n>We operationalize this framework via the Semantic Representation Heuristic Search (SRHS) algorithm, which preserves interpretability and structural coherence of the adversarial prompts.<n>Our framework achieves a 99.71% average attack success rate across 26 open-source LLMs, with strong transferability and stealth.
  arXiv  Detail & Related papers  (2026-05-09T11:43:47Z)
• Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction [51.50282796099369]
  This paper develops a multi-dimensional instruction uncertainty reduction framework to generate semantically constrained adversarial examples.<n>By predicting the language-guided sampling process, the optimization process will be stabilized by the designed ResAdv-DDIM sampler.<n>We realize the reference-free generation of semantically constrained 3D adversarial examples for the first time.
  arXiv  Detail & Related papers  (2025-10-27T04:02:52Z)
• SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations [47.0190003379175]
  Large Language Models (LLMs) are increasingly deployed in high-risk domains.<n>LLMs often produce hallucinations, raising serious concerns about their reliability.<n>We propose Semantically Equivalent and Coherent Attacks (SECA) to elicit hallucinations.
  arXiv  Detail & Related papers  (2025-10-05T23:44:54Z)
• Semantic Representation Attack against Aligned Large Language Models [18.13997425681567]
  Large Language Models (LLMs) increasingly employ alignment techniques to prevent harmful outputs.<n>Current methods typically target exact affirmative responses, suffering from limited convergence, unnatural prompts, and high computational costs.<n>We introduce Semantic Representation Attack, a novel paradigm that fundamentally reconceptualizes adversarial objectives against aligned LLMs.
  arXiv  Detail & Related papers  (2025-09-18T15:06:46Z)
• PromptSleuth: Detecting Prompt Injection via Semantic Intent Invariance [10.105673138616483]
  Large Language Models (LLMs) are increasingly integrated into real-world applications, from virtual assistants to autonomous agents.<n>As attackers evolve with paraphrased, obfuscated, and even multi-task injection strategies, existing benchmarks are no longer sufficient to capture the full spectrum of emerging threats.<n>We propose PromptSleuth, a semantic-oriented defense framework that detects prompt injection by reasoning over task-level intent rather than surface features.
  arXiv  Detail & Related papers  (2025-08-28T15:19:07Z)
• ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models [67.75439511654078]
  Large Vision-Language Models (LVLMs) have introduced a new paradigm for understanding and reasoning about image input through textual responses.<n>They face the persistent challenge of hallucination, which introduces practical weaknesses and raises concerns about their reliable deployment in real-world applications.<n>We propose ONLY, a training-free decoding approach that requires only a single query and a one-layer intervention during decoding, enabling efficient real-time deployment.
  arXiv  Detail & Related papers  (2025-07-01T16:01:08Z)
• REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective [57.57786477441956]
  We propose an adaptive and semantic optimization problem over the population of responses.<n>Our objective doubles the attack success rate (ASR) on Llama3 and increases the ASR from 2% to 50% with circuit breaker defense.
  arXiv  Detail & Related papers  (2025-02-24T15:34:48Z)
• Learning diverse attacks on large language models for robust red-teaming and safety tuning [126.32539952157083]
  Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe deployment of large language models.<n>We show that even with explicit regularization to favor novelty and diversity, existing approaches suffer from mode collapse or fail to generate effective attacks.<n>We propose to use GFlowNet fine-tuning, followed by a secondary smoothing phase, to train the attacker model to generate diverse and effective attack prompts.
  arXiv  Detail & Related papers  (2024-05-28T19:16:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.