Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models

• URL: http://arxiv.org/abs/2604.10697v1
• Date: Sun, 12 Apr 2026 15:46:22 GMT
• Title: Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
• Abstract summary: We propose SinkProbe, a hallucination detection method grounded in the observation that hallucinations are deeply entangled with attention sinks.<n>Our findings yield a novel hallucination detection method grounded in theory that produces state-of-the-art results across popular datasets and LLMs.
• Score: 5.559343107521254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models frequently exhibit hallucinations: fluent and confident outputs that are factually incorrect or unsupported by the input context. While recent hallucination detection methods have explored various features derived from attention maps, the underlying mechanisms they exploit remain poorly understood. In this work, we propose SinkProbe, a hallucination detection method grounded in the observation that hallucinations are deeply entangled with attention sinks - tokens that accumulate disproportionate attention mass during generation - indicating a transition from distributed, input-grounded attention to compressed, prior-dominated computation. Importantly, although sink scores are computed solely from attention maps, we find that the classifier preferentially relies on sinks whose associated value vectors have large norms. Moreover, we show that previous methods implicitly depend on attention sinks by establishing their mathematical relationship to sink scores. Our findings yield a novel hallucination detection method grounded in theory that produces state-of-the-art results across popular datasets and LLMs.

Related papers

• Disentangling Semantic Attention from Structural Bias in the Attention Manifold [61.68044165974505]
  Multimodal Large Language Models (MLLMs) exhibit disproportionate attention toward certain semantically uninformative visual tokens.<n>We introduce Saliency-guided Purification and Adaptive Redistribution (SPAR), a training-free, plug-and-play intervention.
  arXiv  Detail & Related papers  (2026-07-27T05:30:19Z)
• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation [50.51650162235191]
  We propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention.<n>IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.
  arXiv  Detail & Related papers  (2026-04-02T12:51:07Z)
• DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning [48.77865928715759]
  Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models.<n> hallucination remains a critical issue that hinders their reliability.<n>We propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives.
  arXiv  Detail & Related papers  (2026-03-17T12:40:29Z)
• Detecting Contextual Hallucinations in LLMs with Frequency-Aware Attention [27.49425252327799]
  We introduce a frequency-aware perspective on attention by analyzing its variation during generation.<n>We develop a lightweight hallucination detector using high-frequency attention features.
  arXiv  Detail & Related papers  (2026-02-20T11:18:45Z)
• The Map of Misbelief: Tracing Intrinsic and Extrinsic Hallucinations Through Attention Patterns [1.0896567381206717]
  Large Language Models (LLMs) are increasingly deployed in safety-critical domains, yet remain susceptible to hallucinations.<n>We introduce a principled evaluation framework that differentiates between extrinsic and intrinsic hallucination categories.<n>We propose novel attention aggregation strategies that improve both interpretability and hallucination detection performance.
  arXiv  Detail & Related papers  (2025-11-13T22:42:18Z)
• Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluation [9.540386616651295]
  Chain-of-Thought (CoT) prompting can mitigate hallucinations by encouraging step-by-step reasoning.<n>Our study highlights an overlooked trade-off in the use of reasoning.
  arXiv  Detail & Related papers  (2025-06-20T15:49:37Z)
• Attention Head Embeddings with Trainable Deep Kernels for Hallucination Detection in LLMs [47.18623962083962]
  We present a novel approach for detecting hallucinations in large language models.<n>We find that hallucinated responses exhibit smaller deviations from their prompts compared to grounded responses.<n>We propose a model-intrinsic detection method that uses distributional distances as principled hallucination scores.
  arXiv  Detail & Related papers  (2025-06-11T15:59:15Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• Hallucinated Span Detection with Multi-View Attention Features [8.747292152322578]
  This study addresses the problem of hallucinated span detection in the outputs of large language models.<n>It has received less attention than output-level hallucination detection despite its practical importance.
  arXiv  Detail & Related papers  (2025-04-06T03:00:58Z)
• Hallucination Detection in LLMs Using Spectral Features of Attention Maps [7.034766253049102]
  Large Language Models (LLMs) have demonstrated remarkable performance across various tasks but remain prone to hallucinations.<n>Recent methods leverage attention map properties to this end, though their effectiveness remains limited.<n>We propose the $textLapEigvals$ method, which uses the top-$k$ eigenvalues of the Laplacian matrix derived from the attention maps as an input to hallucination detection probes.
  arXiv  Detail & Related papers  (2025-02-24T19:30:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.