Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations

• URL: http://arxiv.org/abs/2604.04743v1
• Date: Mon, 06 Apr 2026 15:08:54 GMT
• Title: Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations
• Abstract summary: We show that hallucinations arise from task-dependent basin structure in latent space.<n>We formalize this behavior with task-complexity and multi-basin theorems, characterize basin emergence in L-layer transformers, and show that geometry-aware steering can reduce hallucination probability without retraining.
• Score: 13.629487815782355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) hallucinate: they produce fluent outputs that are factually incorrect. We present a geometric dynamical systems framework in which hallucinations arise from task-dependent basin structure in latent space. Using autoregressive hidden-state trajectories across multiple open-source models and benchmarks, we find that separability is strongly task-dependent rather than universal: factoid settings can show clearer basin separation, whereas summarization and misconception-heavy settings are typically less stable and often overlap. We formalize this behavior with task-complexity and multi-basin theorems, characterize basin emergence in L-layer transformers, and show that geometry-aware steering can reduce hallucination probability without retraining.

Related papers

• Why LLMs Hallucinate on Structured Knowledge: A Mechanistic Analysis of Reasoning over Linearized Representations [36.45654492179688]
  We find that hallucinations arise from systematic internal dynamics rather than random noise.<n>First, attention disproportionately concentrates toward shortcut-like structural cues rather than distributing across the full context.<n>Second, feed-forward representations fail to ground the provided knowledge, causing the model to revert to parametric memory.
  arXiv  Detail & Related papers  (2026-05-25T22:08:59Z)
• Locate-then-Sparsify: Attribution Guided Sparse Strategy for Visual Hallucination Mitigation [68.41785694664011]
  We propose a plug-and-play framework called Locate-Then-Sparsify for Feature Steering (LTS-FS)<n>It controls the steering intensity according to the hallucination relevance of each layer.<n>Our framework effectively mitigates hallucination while preserving strong performance.
  arXiv  Detail & Related papers  (2026-03-17T09:16:50Z)
• Lyapunov Probes for Hallucination Detection in Large Foundation Models [27.51972811669081]
  hallucinations tend to arise at the boundaries of knowledge-transition regions separating stable and unstable zones.<n>We propose Lyapunov Probes: lightweight networks trained with derivative-based stability constraints.<n>Experiments on diverse datasets and models demonstrate consistent improvements over existing baselines.
  arXiv  Detail & Related papers  (2026-03-06T09:32:59Z)
• HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs [17.130698952440316]
  hallucinations in large language models (MLLMs) have become increasingly crucial in the model development process.<n>We propose the hypothesis that hallucination manifests as measurable irregularities within the internal layer dynamics of MLLMs.<n>By incorporating such modifications, textsctextscHalluShift++ broadens the efficacy of hallucination detection from text-based large language models.
  arXiv  Detail & Related papers  (2025-12-08T16:24:46Z)
• Mitigating Diffusion Model Hallucinations with Dynamic Guidance [75.90458880492054]
  Diffusion models often produce hallucinatory samples with structural inconsistencies that lie outside of the support of the true data distribution.<n>We introduce Dynamic Guidance, which mitigates hallucinations by selectively sharpening the score function only along the pre-determined directions known to cause artifacts.<n>To our knowledge, this is the first approach that addresses hallucinations at generation time rather than through post-hoc filtering.
  arXiv  Detail & Related papers  (2025-10-06T20:31:13Z)
• Review of Hallucination Understanding in Large Language and Vision Models [65.29139004945712]
  We present a framework for characterizing both image and text hallucinations across diverse applications.<n>Our investigations reveal that hallucinations often stem from predictable patterns in data distributions and inherited biases.<n>This survey provides a foundation for developing more robust and effective solutions to hallucinations in real-world generative AI systems.
  arXiv  Detail & Related papers  (2025-09-26T09:23:08Z)
• Grounding the Ungrounded: A Spectral-Graph Framework for Quantifying Hallucinations in Multimodal LLMs [19.099044165107696]
  We present a rigorous, information-geometric framework in diffusion dynamics that quantifies hallucination in MLLMs.<n>We derive Courant--Fischer bounds on a temperature-dependent hallucination energy and use RKHS eigenmodes to obtain modality-aware, interpretable measures.
  arXiv  Detail & Related papers  (2025-08-26T18:54:52Z)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM [58.2298313720146]
  Multimodal hallucinations are multi-sourced and arise from diverse causes.<n>Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations.
  arXiv  Detail & Related papers  (2025-05-30T05:54:36Z)
• Triggering Hallucinations in LLMs: A Quantitative Study of Prompt-Induced Hallucination in Large Language Models [0.0]
  Hallucinations in large language models (LLMs) present a growing challenge across real-world applications.<n>We propose a prompt-based framework to systematically trigger and quantify hallucination.
  arXiv  Detail & Related papers  (2025-05-01T14:33:47Z)
• Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations [82.42811602081692]
  This paper introduces a subsequence association framework to systematically trace and understand hallucinations.<n>Key insight is hallucinations that arise when dominant hallucinatory associations outweigh faithful ones.<n>We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts.
  arXiv  Detail & Related papers  (2025-04-17T06:34:45Z)
• HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data [102.56792377624927]
  hallucinations inherent in machine-generated data remain under-explored. We present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. Our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA.
  arXiv  Detail & Related papers  (2023-11-22T04:52:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.