When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs

• URL: http://arxiv.org/abs/2609.15106v1
• Date: Mon, 14 Sep 2026 06:35:28 GMT
• Title: When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs
• Abstract summary: Large language models can hallucinate even when the knowledge required for a correct answer is already available.<n>We show that model predictions can be highly sensitive to individual query keywords.<n>We introduce a two-stage keyword-perturbation method for hallucination detection.
• Score: 6.246552320858934
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models can hallucinate even when the knowledge required for a correct answer is already available. We study this failure through a latent-key view of inference, where answer selection depends on competition among associations acquired during pretraining. We show that model predictions can be highly sensitive to individual query keywords, that these influential keywords exhibit entity-specific binding, and that their effects are systematically shaped by pretraining frequency. Multiple bindings can also compete and exhibit higher-order interactions within the same query. Based on this mechanism, we introduce a two-stage keyword-perturbation method for hallucination detection. By removing influential keywords and measuring how the model reorganizes its prediction, the method distinguishes errors caused by misleading key associations from correct decisions supported by diagnostic evidence. Across multiple models and benchmarks, perturbation provides a strong and transferable detection signal, reaching $.910$ AUROC on probe-known ScientistQA. Finally, we extend the same probabilistic framework to four hallucination regimes: knowledge deficit, wrong knowledge, context distraction, and unstable inference. Their operational distributions across benchmarks provide diagnostic context for why different detector families succeed in different settings.

Related papers

• HIVE: Understanding Post-Hallucination Reasoning in Vision Language Models [70.67112531913669]
  Hallucinations in vision language models are commonly treated as semantic errors.<n>Prior work mainly focuses on detecting or suppressing hallucinations at generation time.<n>We study Post Hallucination Reasoning (PHR), the stage in which hallucinated semantics enter the model's inference context.
  arXiv  Detail & Related papers  (2026-07-08T15:02:11Z)
• Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors [11.702863087019809]
  Large language models often produce hallucinated answers that violate prompt-level constraints.<n>We study inference misalignment: a mismatch between the answer supported by the prompt and the answer favored by statistically salient latent associations.<n>We formalize this view with a latent key-task model, in which pretraining-frequency imbalance can cause a shortcut path to dominate the constraint-sensitive path.
  arXiv  Detail & Related papers  (2026-07-01T05:02:43Z)
• Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models [54.041320081289996]
  We present the first systematic empirical study of uncertainty estimation in audio-aware large language models (ALLMs)<n>We benchmark five representative methods, including predictive entropy, length-normalized entropy, semantic entropy, discrete semantic entropy, and P(True)<n>Our results reveal two key findings. First, semantic-level and verification-based methods consistently outperform token-level baselines on general audio reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-04-28T12:56:22Z)
• Causality is Key for Interpretability Claims to Generalise [35.833847356014154]
  Interpretability research on large language models (LLMs) has yielded important insights into model behaviour.<n> recurring pitfalls persist: findings that do not generalise, and causal interpretations that outrun the evidence.<n>Pearl's causal hierarchy clarifies what an interpretability study can justify.
  arXiv  Detail & Related papers  (2026-02-18T18:45:04Z)
• Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know" [47.930782177987446]
  Large language models often struggle to recognize their knowledge limits in closed-book question answering, leading to confident hallucinations.<n>We evaluate three task-equivalent prompting regimes: Direct, Assistive, and Incremental, across different model scales and multi-hop QA benchmarks.<n>Because factual knowledge is stable while hallucinations are agreement, cross-regime provides a precise signal of internal uncertainty.
  arXiv  Detail & Related papers  (2026-02-04T18:39:58Z)
• Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts [74.47786985522762]
  We identify a critical failure mode termed textual inertia, where models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence.<n>We propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the reasoning chains of diverse LMMs.<n>Results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation.
  arXiv  Detail & Related papers  (2026-01-07T16:39:34Z)
• Prompt-Response Semantic Divergence Metrics for Faithfulness Hallucination and Misalignment Detection in Large Language Models [0.0]
  This paper introduces Semantic Divergence Metrics (SDM), a novel framework for detecting Faithfulness Hallucinations.<n>A heatmap of topic co-occurances between prompts and responses can be viewed as a quantified two-dimensional visualization of the user-machine dialogue.
  arXiv  Detail & Related papers  (2025-08-13T20:55:26Z)
• Detecting Token-Level Hallucinations Using Variance Signals: A Reference-Free Approach [0.0]
  Large Language Models (LLMs) have demonstrated impressive generative capabilities across diverse tasks but remain susceptible to hallucinations.<n>We introduce a reference-free, token-level hallucination detection framework that leverages the variance in token log-probabilities across multiple generations.<n>Our approach is model-agnostic, interpretable, and suited for real-time or post-hoc analysis.
  arXiv  Detail & Related papers  (2025-07-05T19:20:59Z)
• Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation [9.540386616651295]
  Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning.<n>Our study highlights an overlooked trade-off in the use of reasoning.
  arXiv  Detail & Related papers  (2025-06-20T15:49:37Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking [55.2155025063668]
  KnowHalu is a novel approach for detecting hallucinations in text generated by large language models (LLMs) It uses step-wise reasoning, multi-formulation query, multi-form knowledge for factual checking, and fusion-based detection mechanism. Our evaluations demonstrate that KnowHalu significantly outperforms SOTA baselines in detecting hallucinations across diverse tasks.
  arXiv  Detail & Related papers  (2024-04-03T02:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.