DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

• URL: http://arxiv.org/abs/2603.16459v1
• Date: Tue, 17 Mar 2026 12:40:29 GMT
• Title: DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning
• Abstract summary: Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models.<n> hallucination remains a critical issue that hinders their reliability.<n>We propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives.
• Score: 48.77865928715759
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty (e.g., entropy) has been widely used as an effective signal to indicate potential factual errors. Nevertheless, the fixed-length generation paradigm of D-LLMs implies that tokens contribute unevenly to hallucination detection, with only a small subset providing meaningful signals. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide important signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To address the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by filtering out non-informative tokens and emphasizing semantically meaningful ones. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models.

Related papers

• DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Models [59.414664850109155]
  Diffusion large language models (D-LLMs) have emerged as a promising paradigm for text generation.<n>D-LLMs remain vulnerable to hallucinations, where fluent outputs may contain factually incorrect or unsupported content.
  arXiv  Detail & Related papers  (2026-07-24T06:45:43Z)
• TRE: Training-Free Hallucination Detection for Diffusion Language Models [9.622962795441852]
  We propose TRE, a training-free hallucination detection metric for D-LLMs.<n> TRE estimates hallucination risk directly from the entropy signals of a single generation, without requiring any detector training or repeated sampling.<n>Extensive experiments on multiple D-LLMs and QA datasets demonstrate that TRE achieves competitive performance, while enjoying strong generalizability, efficiency, and robustness.
  arXiv  Detail & Related papers  (2026-06-29T00:47:48Z)
• Lost in Diffusion: Uncovering Hallucination Patterns and Failure Modes in Diffusion Large Language Models [10.020835642521897]
  We present the first controlled comparative study to evaluate hallucination patterns in dLLMs.<n>Our results demonstrate that current dLLMs exhibit a higher propensity for hallucination than AR counterparts controlled for architecture, scale, and pre-training weights.<n>Our findings underscore that although dLLMs have narrowed the performance gap on general tasks, their distinct hallucination mechanisms pose a critical challenge to model reliability.
  arXiv  Detail & Related papers  (2026-04-12T09:59:41Z)
• Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction [49.96701537295129]
  Large Vision-Language Models (LVLMs) have achieved remarkable success across cross-modal tasks but remain hindered by hallucinations.<n>Existing methods mitigate hallucinations but often alter generation behavior, resulting in shorter outputs and shifted token distributions.<n>We propose MESA, an effective plug-and-play framework that performs controlled and selective latent intervention for hallucination mitigation.
  arXiv  Detail & Related papers  (2026-04-09T07:31:27Z)
• TDGNet: Hallucination Detection in Diffusion Language Models via Temporal Dynamic Graphs [30.313604786976715]
  Diffusion language models (D-LLMs) offer parallel denoising and bidirectional context.<n> hallucination detection for D-LLMs remains underexplored.<n>We introduce TDGNet, a temporal dynamic graph framework that formulates hallucination detection as learning over evolving token-level attention graphs.
  arXiv  Detail & Related papers  (2026-02-08T16:35:30Z)
• Seeing Through the Chain: Mitigate Hallucination in Multimodal Reasoning Models via CoT Compression and Contrastive Preference Optimization [78.94590726578014]
  multimodal reasoning models (MLRMs) remain prone to hallucinations, and effective solutions are still underexplored.<n>We propose C3PO, a training-based mitigation framework comprising textbfCompression and textbfPreference textbfOptimization.
  arXiv  Detail & Related papers  (2026-02-03T11:00:55Z)
• EDIS: Diagnosing LLM Reasoning via Entropy Dynamics [3.858418431840288]
  We show that the emphtemporal evolution of confidence during generation carries richer information than aggregate statistics alone.<n>We introduce the Entropy Dynamics Instability Score (textbfEDIS), a trajectory-level metric quantifying instability in entropy evolution.
  arXiv  Detail & Related papers  (2026-02-01T15:43:50Z)
• From Detection to Diagnosis: Advancing Hallucination Analysis with Automated Data Synthesis [7.3390265169910665]
  Hallucinations in Large Language Models (LLMs) represent a core obstacle to their reliable deployment in critical domains.<n>To address this limitation, a new research paradigm is proposed, shifting from "detection" to "diagnosis"
  arXiv  Detail & Related papers  (2025-12-31T05:06:35Z)
• TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models [49.83690850047884]
  hallucination problem in D-LLMs remains underexplored, limiting their reliability in real-world applications.<n>Existing hallucination detection methods are designed for AR-LLMs and rely on signals from single-step generation.<n>We propose TraceDet, a novel framework that explicitly leverages the intermediate denoising steps of D-LLMs for hallucination detection.
  arXiv  Detail & Related papers  (2025-09-30T02:01:10Z)
• Spatial Reasoning with Denoising Models [49.83744014336816]
  We introduce a framework to perform reasoning over sets of continuous variables via denoising generative models.<n>For the first time, that order of generation can successfully be predicted by the denoising network itself.<n>Using these findings, we can increase the accuracy of specific reasoning tasks from 1% to >50%.
  arXiv  Detail & Related papers  (2025-02-28T14:08:30Z)
• Enhancing Hallucination Detection through Noise Injection [9.582929634879932]
  Large Language Models (LLMs) are prone to generating plausible yet incorrect responses, known as hallucinations.<n>We show that detection can be improved significantly by taking into account model uncertainty in the Bayesian sense.<n>We propose a very simple and efficient approach that perturbs an appropriate subset of model parameters, or equivalently hidden unit activations, during sampling.
  arXiv  Detail & Related papers  (2025-02-06T06:02:20Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• The role of noise in denoising models for anomaly detection in medical images [62.0532151156057]
  Pathological brain lesions exhibit diverse appearance in brain images. Unsupervised anomaly detection approaches have been proposed using only normal data for training. We show that optimization of the spatial resolution and magnitude of the noise improves the performance of different model training regimes.
  arXiv  Detail & Related papers  (2023-01-19T21:39:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.