Sanity Checks for Long-Form Hallucination Detection

• URL: http://arxiv.org/abs/2605.08346v1
• Date: Fri, 08 May 2026 18:00:58 GMT
• Title: Sanity Checks for Long-Form Hallucination Detection
• Abstract summary: Hallucination detection methods for large language models operate on chain-of-thought reasoning traces, yet it remains unclear whether they evaluate the reasoning itself or merely exploit surface correlates of the final answer.<n>We introduce a controlled-invariance methodology that exposes this distinction through two oracle tests: textscForce, which replaces each response's final answer with the ground truth while preserving the reasoning trace, and textscRemove, which strips answer-announcement steps while leaving the trajectory intact.<n>This reveals if their predictive power derives from answer-level artifacts rather than from the structure
• Score: 6.501493411588847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hallucination detection methods for large language models increasingly operate on chain-of-thought reasoning traces, yet it remains unclear whether they evaluate the reasoning itself or merely exploit surface correlates of the final answer. We introduce a controlled-invariance methodology that exposes this distinction through two oracle tests: \textsc{Force}, which replaces each response's final answer with the ground truth while preserving the reasoning trace, and \textsc{Remove}, which strips answer-announcement steps while leaving the trajectory intact. This reveals if their predictive power derives from answer-level artifacts rather than from the structure or validity of intermediate reasoning. We further show that once these artifacts are controlled for, effective detection does not necessarily require complex learned representations: TRACT, a lightweight scorer built on lexical trajectory features (hedging trends, step-length dynamics, and cross-response vocabulary convergence), achieves strong robustness while remaining competitive with or outperforming existing baselines on unperturbed traces. These findings suggest that the current central challenge in reasoning-aware hallucination detection is not the absence of signal in the trace, but the failure to isolate it from endpoint cues.

Related papers

• Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning [109.33717747526583]
  Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks.<n>We introduce textbfCaRL (textbfCapability-textbfaligned textbfReinforcement textbfL) to align behavior with capability boundaries.<n>Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties.
  arXiv  Detail & Related papers  (2026-07-31T09:30:33Z)
• Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models [14.440332703001218]
  We propose REDE, a learning framework for denoising reasoning traces for hallucination detection.<n>Rede uses final-answer attention as an automatic supervision signal to shape the step-level representation space.<n>Experiments show that REDE consistently improves hallucination detection performance over competitive baselines.
  arXiv  Detail & Related papers  (2026-07-24T08:48:17Z)
• Scalable Token-Level Hallucination Detection in Large Language Models [63.3426544914783]
  Internal hallucinations are difficult to detect in reasoning-intensive tasks.<n>We propose TokenHD, a holistic pipeline for training token-level hallucination detectors.
  arXiv  Detail & Related papers  (2026-05-12T16:47:40Z)
• Neural Uncertainty Principle: A Unified View of Adversarial Fragility and LLM Hallucination [60.197429875410286]
  Adversarial vulnerability in vision and hallucination in large language models are conventionally viewed as separate problems.<n>Formalizing a Neural Uncertainty Principle (NUP) under a loss-induced state, we find that in near-bound regimes, further compression must be accompanied by increased sensitivity dispersion.<n>In vision, masking highly coupled components improves robustness without costly adversarial training.<n>In language, the same prefill-stage probe detects hallucination risk before generating any answer tokens.
  arXiv  Detail & Related papers  (2026-03-20T02:07:10Z)
• DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning [48.77865928715759]
  Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models.<n> hallucination remains a critical issue that hinders their reliability.<n>We propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives.
  arXiv  Detail & Related papers  (2026-03-17T12:40:29Z)
• The Phenomenology of Hallucinations [2.5782420501870296]
  We show that language models hallucinate not because they fail to detect uncertainty, but because of a failure to integrate it into output generation.<n>Across architectures, uncertain inputs are reliably identified, occupying high-dimensional regions with 2-3$times$ the intrinsic dimensionality of factual inputs.<n>However, this internal signal is weakly coupled to the output layer: uncertainty migrates into low-sensitivity subspaces, becoming geometrically amplified yet functionally silent.
  arXiv  Detail & Related papers  (2026-03-14T11:55:55Z)
• Harnessing Reasoning Trajectories for Hallucination Detection via Answer-agreement Representation Shaping [31.704726867711955]
  We introduce Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations.<n>ARS generates counterfactual answers through small latent interventions.<n>ARS consistently improves detection and achieves substantial gains over strong baselines.
  arXiv  Detail & Related papers  (2026-01-24T13:47:51Z)
• Unsupervised Hallucination Detection by Inspecting Reasoning Processes [53.15199932086543]
  Unsupervised hallucination detection aims to identify hallucinated content generated by large language models (LLMs) without relying on labeled data.<n>We propose IRIS, an unsupervised hallucination detection framework, leveraging internal representations intrinsic to factual correctness.<n>Our approach is fully unsupervised, computationally low cost, and works well even with few training data, making it suitable for real-time detection.
  arXiv  Detail & Related papers  (2025-09-12T06:58:17Z)
• Semantic Energy: Detecting LLM Hallucination Beyond Entropy [106.92072182161712]
  Large Language Models (LLMs) are being increasingly deployed in real-world applications, but they remain susceptible to hallucinations.<n>Uncertainty estimation is a feasible approach to detect such hallucinations.<n>We introduce Semantic Energy, a novel uncertainty estimation framework.
  arXiv  Detail & Related papers  (2025-08-20T07:33:50Z)
• Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models [12.270274049887298]
  Reasoning traces can be redundant or logically inconsistent, making them a new source of hallucination.<n>Existing hallucination detection methods focus primarily on answer-level uncertainty.<n>We propose RACE, a novel framework specifically tailored for hallucination detection in LRMs.
  arXiv  Detail & Related papers  (2025-06-05T09:54:04Z)
• RePPL: Recalibrating Perplexity by Uncertainty in Semantic Propagation and Language Generation for Explainable QA Hallucination Detection [26.186204911845866]
  hallucinations remain a vital obstacle to large language models' trustworthy use.<n>We propose RePPL to recalibrate uncertainty measurement by these two aspects.<n>Our method achieves the best comprehensive detection performance across various QA datasets.
  arXiv  Detail & Related papers  (2025-05-21T11:23:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.