Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer

• URL: http://arxiv.org/abs/2605.22007v1
• Date: Thu, 21 May 2026 05:08:37 GMT
• Title: Hallucination as Commitment Failure: Larger LLMs Misfire Despite Knowing the Answer
• Abstract summary: We introduce a semantic notion of answer availability that aggregates token-level variants expressing the same answer concept.<n>We ask whether the correct concept is already available at the moment the model commits to an answer.<n>Across Qwen and Llama models from 0.8B to 72B in both Instruct and Base variants, 16-47% of Instruct hallucinations occur with substantial probability mass already on the correct concept.
• Score: 8.883246421566243
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hallucination is often viewed as a direct consequence of missing knowledge: a model answers incorrectly when the correct answer is absent from its generation-time distribution, and correctly when it is present. We test this assumption by introducing a semantic notion of answer availability that aggregates token-level variants expressing the same answer concept, and asks whether the correct concept is already available at the moment the model commits to an answer. Across Qwen and Llama models from 0.8B to 72B in both Instruct and Base variants, 16-47% of Instruct hallucinations occur with substantial probability mass already on the correct concept, and the rate rises monotonically with scale. Comparing such failures against correct generations with matched semantic support, the distinguishing factor is not whether the correct concept is represented, but how its probability is distributed: correct generations concentrate mass on a single surface form, hallucinations disperse it across alternatives. The same sharpening asymmetry extends across multi-token generation and is detectable in pre-generation hidden states. Together, these results identify a single mechanism: instruction tuning sharpens answer commitment with scale, making helpfulness and confident hallucination two consequences of the same underlying disposition.

Related papers

• Understanding Why Language Models Hallucinate: Testing Reasoning Against Priors [11.702863087019809]
  Large language models often produce hallucinated answers that violate prompt-level constraints.<n>We study inference misalignment: a mismatch between the answer supported by the prompt and the answer favored by statistically salient latent associations.<n>We formalize this view with a latent key-task model, in which pretraining-frequency imbalance can cause a shortcut path to dominate the constraint-sensitive path.
  arXiv  Detail & Related papers  (2026-07-01T05:02:43Z)
• Score-Control for Hallucination Reduction in Diffusion Models [5.244752390553863]
  Diffusion models have emerged as the backbone of modern generative AI, powering advances in vision, language, audio and other modalities.<n>Despite their success, they suffer from hallucinations, implausible samples that lie outside the support of true data distribution.<n>We first empirically confirm previously proposed hypothesis that score smoothness causes hallucinations in Image Generation diffusion models and provide a density-based perspective.<n>Motivated by this, we introduce a Variance-Guided Score Modulation (VSM) strategy that controls the score Jacobian, in turn reducing score smoothness and better approxing the ground truth score that decreases hallucinations.
  arXiv  Detail & Related papers  (2026-05-29T21:35:54Z)
• Feeling the Strength but Not the Source: Partial Introspection in LLMs [0.0]
  Anthropic claims frontier models can sometimes detect and name injected "concepts" represented as activation directions.<n>We reproduce Anthropic's multi-turn "emergent introspection" result on Meta-Llama-3.1-8B-Instruct.<n>We find that introspection is not exclusive to very large or capable models.
  arXiv  Detail & Related papers  (2025-12-13T17:51:13Z)
• Counting Hallucinations in Diffusion Models [34.45858211220468]
  Diffusion probabilistic models (DPMs) have demonstrated remarkable progress in generative tasks, such as image and video synthesis.<n>They often produce hallucinated samples (hallucinations) that conflict with real-world knowledge.<n>Despite their prevalence, the lack of feasible methodologies for systematically quantifying such hallucinations hinders progress.
  arXiv  Detail & Related papers  (2025-10-15T01:48:04Z)
• Mitigating Diffusion Model Hallucinations with Dynamic Guidance [75.90458880492054]
  Diffusion models often produce hallucinatory samples with structural inconsistencies that lie outside of the support of the true data distribution.<n>We introduce Dynamic Guidance, which mitigates hallucinations by selectively sharpening the score function only along the pre-determined directions known to cause artifacts.<n>To our knowledge, this is the first approach that addresses hallucinations at generation time rather than through post-hoc filtering.
  arXiv  Detail & Related papers  (2025-10-06T20:31:13Z)
• RePPL: Recalibrating Perplexity by Uncertainty in Semantic Propagation and Language Generation for Explainable QA Hallucination Detection [26.186204911845866]
  hallucinations remain a vital obstacle to large language models' trustworthy use.<n>We propose RePPL to recalibrate uncertainty measurement by these two aspects.<n>Our method achieves the best comprehensive detection performance across various QA datasets.
  arXiv  Detail & Related papers  (2025-05-21T11:23:05Z)
• Counterfactual Realizability [52.85109506684737]
  We introduce a formal definition of realizability, the ability to draw samples from a distribution, and then develop a complete algorithm to determine whether an arbitrary counterfactual distribution is realizable.<n>We illustrate the implications of this new framework for counterfactual data collection using motivating examples from causal fairness and causal reinforcement learning.
  arXiv  Detail & Related papers  (2025-03-14T20:54:27Z)
• Distinguishing Ignorance from Error in LLM Hallucinations [43.62904897907926]
  We distinguish between two types of hallucinations: ones where the model does not hold the correct answer in its parameters, which we term HK-, and ones where the model answers incorrectly despite having the required knowledge, termed HK+.<n>We show that different models hallucinate on different examples, which motivates constructing model-specific hallucination datasets for training detectors.
  arXiv  Detail & Related papers  (2024-10-29T14:31:33Z)
• To Believe or Not to Believe Your LLM [51.2579827761899]
  We explore uncertainty quantification in large language models (LLMs) We derive an information-theoretic metric that allows to reliably detect when only epistemic uncertainty is large. We conduct a series of experiments which demonstrate the advantage of our formulation.
  arXiv  Detail & Related papers  (2024-06-04T17:58:18Z)
• On Large Language Models' Hallucination with Regard to Known Facts [74.96789694959894]
  Large language models are successful in answering factoid questions but are also prone to hallucination. We investigate the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from the perspective of inference dynamics. Our study shed light on understanding the reasons for LLMs' hallucinations on their known facts, and more importantly, on accurately predicting when they are hallucinating.
  arXiv  Detail & Related papers  (2024-03-29T06:48:30Z)
• A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning [68.76846801719095]
  We show that double descent appears exactly when and where it occurs, and that its location is not inherently tied to the threshold p=n. This provides a resolution to tensions between double descent and statistical intuition.
  arXiv  Detail & Related papers  (2023-10-29T12:05:39Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.