Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations

• URL: http://arxiv.org/abs/2605.26362v1
• Date: Mon, 25 May 2026 22:08:59 GMT
• Title: Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations
• Abstract summary: We find that hallucinations arise from systematic internal dynamics rather than random noise.<n>First, attention disproportionately concentrates toward shortcut-like structural cues rather than distributing across the full context.<n>Second, feed-forward representations fail to ground the provided knowledge, causing the model to revert to parametric memory.
• Score: 36.45654492179688
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In many reasoning tasks, large language models (LLMs) rely on structured external knowledge, such as graphs and tables, which is typically linearized into sequential token representations. However, even when sufficient knowledge is available, LLMs can still produce hallucinated outputs, and the underlying mechanisms behind such failures remain poorly understood. We investigate these mechanisms and find that hallucinations arise from systematic internal dynamics rather than random noise. First, attention disproportionately concentrates toward shortcut-like structural cues rather than distributing across the full context. Second, feed-forward representations fail to ground the provided knowledge, causing the model to revert to parametric memory. Moreover, our results indicate that hallucination is consistently associated with failures in semantic grounding within feed-forward layers, while attention allocation exhibits greater task-dependent variability. Finally, we show that these mechanistic patterns generalize beyond single-hop graphs to multi-hop and tabular settings, enabling effective hallucination detection across structured knowledge formats.

Related papers

• Disentangling Semantic Attention from Structural Bias in the Attention Manifold [61.68044165974505]
  Multimodal Large Language Models (MLLMs) exhibit disproportionate attention toward certain semantically uninformative visual tokens.<n>We introduce Saliency-guided Purification and Adaptive Redistribution (SPAR), a training-free, plug-and-play intervention.
  arXiv  Detail & Related papers  (2026-07-27T05:30:19Z)
• SLASH the Sink: Sharpening Structural Attention Inside LLMs [55.75072688988806]
  Large Language Models (LLMs) show remarkable semantic understanding but struggle with structural understanding when processing graph topologies in a serialized format.<n>We propose StructuraL Attention SHarpening (SLASH), which amplifies this internal structural understanding via a plug-and-play attention redistribution.
  arXiv  Detail & Related papers  (2026-05-11T12:59:07Z)
• Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations [13.629487815782355]
  We show that hallucinations arise from task-dependent basin structure in latent space.<n>We formalize this behavior with task-complexity and multi-basin theorems, characterize basin emergence in L-layer transformers, and show that geometry-aware steering can reduce hallucination probability without retraining.
  arXiv  Detail & Related papers  (2026-04-06T15:08:54Z)
• When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression [41.881697211314396]
  Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge.<n>We show that reasoning hallucinations arise from two fundamental mechanisms: textbfPath Reuse, where memorized knowledge overrides contextual constraints during early training, and textbfPath Compression, where frequently traversed multi-step paths collapse into shortcut edges in later training.
  arXiv  Detail & Related papers  (2026-04-04T02:51:46Z)
• Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment [36.45654492179688]
  Graph-based Retrieval-Augmented Generation (GraphRAG) enhances Large Language Models (LLMs)<n>LLMs struggle to interpret the relational and topological information in inputs, resulting in hallucinations that are inconsistent with the retrieved knowledge.<n>We propose two lightweight interpretability metrics: Path Reliance Degree (PRD), which measures over-reliance on shortest-path triples, and Semantic Alignment Score (SAS), which assesses how well the model's internal representations align with the retrieved knowledge.
  arXiv  Detail & Related papers  (2025-12-09T21:52:50Z)
• Review of Hallucination Understanding in Large Language and Vision Models [65.29139004945712]
  We present a framework for characterizing both image and text hallucinations across diverse applications.<n>Our investigations reveal that hallucinations often stem from predictable patterns in data distributions and inherited biases.<n>This survey provides a foundation for developing more robust and effective solutions to hallucinations in real-world generative AI systems.
  arXiv  Detail & Related papers  (2025-09-26T09:23:08Z)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM [58.2298313720146]
  Multimodal hallucinations are multi-sourced and arise from diverse causes.<n>Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations.
  arXiv  Detail & Related papers  (2025-05-30T05:54:36Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• LLMs Will Always Hallucinate, and We Need to Live With This [1.3810901729134184]
  This work argues that hallucinations in language models are not just occasional errors but an inevitable feature of these systems. It is, therefore, impossible to eliminate them through architectural improvements, dataset enhancements, or fact-checking mechanisms.
  arXiv  Detail & Related papers  (2024-09-09T16:01:58Z)
• Knowledge Verification to Nip Hallucination in the Bud [69.79051730580014]
  We demonstrate the feasibility of mitigating hallucinations by verifying and minimizing the inconsistency between external knowledge present in the alignment data and the intrinsic knowledge embedded within foundation LLMs. We propose a novel approach called Knowledge Consistent Alignment (KCA), which employs a well-aligned LLM to automatically formulate assessments based on external knowledge. We demonstrate the superior efficacy of KCA in reducing hallucinations across six benchmarks, utilizing foundation LLMs of varying backbones and scales.
  arXiv  Detail & Related papers  (2024-01-19T15:39:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.