Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals

• URL: http://arxiv.org/abs/2608.17687v1
• Date: Tue, 18 Aug 2026 12:00:20 GMT
• Title: Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signals
• Abstract summary: We introduce InnerExpert, the first method to leverage MoE-specific signals for per-token hallucination detection.<n>Our results show that InnerExpert outperforms existing methods across five datasets and two MoE architectures.
• Score: 2.2302915692528367
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite their widespread use, Large Language Models (LLMs) remain limited by a fundamental problem: the generation of plausible but false content, known as hallucinations. Most existing detection methods operate at the answer or sentence level, yet per-token detection is essential for localizing hallucinated spans and enabling fine-grained interventions. In this paper, we explore the use of the Mixture-of-Experts (MoE) paradigm to address this gap. In MoE architectures, a single forward pass activates a sparse subset of experts (i.e., distinct feedforward networks per layer) via a routing mechanism, producing internal signals (e.g., router entropy, expert disagreement, and expert usage patterns) that are unavailable in dense architectures and have not been previously exploited for hallucination detection. To this end, we introduce InnerExpert, the first method to leverage these MoE-specific signals for per-token hallucination detection. InnerExpert combines routing-level and standard transformer signals into compact per-token feature vectors, classified by a lightweight detector trained on labels produced by an LLM-as-a-judge pipeline, which enables continuous model updates without manual annotation. Our results show that InnerExpert outperforms existing methods across five datasets and two MoE architectures, achieving up to 0.91 answer-level and 0.76 token-level AUROC, while requiring only a single forward pass.

Related papers

• UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations [61.85916280854257]
  Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input.<n>We introduce textbfUniProbe, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass.
  arXiv  Detail & Related papers  (2026-08-11T12:01:59Z)
• DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Models [59.414664850109155]
  Diffusion large language models (D-LLMs) have emerged as a promising paradigm for text generation.<n>D-LLMs remain vulnerable to hallucinations, where fluent outputs may contain factually incorrect or unsupported content.
  arXiv  Detail & Related papers  (2026-07-24T06:45:43Z)
• TRE: Training-Free Hallucination Detection for Diffusion Language Models [9.622962795441852]
  We propose TRE, a training-free hallucination detection metric for D-LLMs.<n> TRE estimates hallucination risk directly from the entropy signals of a single generation, without requiring any detector training or repeated sampling.<n>Extensive experiments on multiple D-LLMs and QA datasets demonstrate that TRE achieves competitive performance, while enjoying strong generalizability, efficiency, and robustness.
  arXiv  Detail & Related papers  (2026-06-29T00:47:48Z)
• Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models [51.099179436350376]
  Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding.<n>We propose Deep Visual Residual MLLM (Deep-VRM) to preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer.<n>This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input.
  arXiv  Detail & Related papers  (2026-06-14T16:05:45Z)
• OpenHalDet: A Unified Benchmark for Hallucination Detection across Diverse Generation Scenarios [34.58119255209895]
  Hallucination detection is essential for the reliable deployment of large language models.<n>We introduce OpenHalDet, a unified benchmark for hallucination detection across diverse generation scenarios.
  arXiv  Detail & Related papers  (2026-06-05T06:38:40Z)
• The Regularizing Power of Language-Training Deepfake Detectors [57.529452351298495]
  Deepfake detection generally suffers from overfitting to low-level domain-specific artifacts.<n>We use a dual-encoder architecture, pairing a frozen specialist detector with a LoRA-tuned MLLM encoder.<n>We employ a reinforcement learning stage that encourages the model to generate descriptive reasoning before classifying.
  arXiv  Detail & Related papers  (2026-05-29T12:01:17Z)
• RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetry [24.677169755426906]
  We propose RouteScan, a non-intrusive auditing framework for detecting harmful behaviors through expert routing telemetry.<n>We show that RouteScan achieves strong generalization, with an AUROC exceeding 0.93 on unseen harmful domains and 0.96 under novel jailbreak wrappers.
  arXiv  Detail & Related papers  (2026-05-24T02:06:44Z)
• Scalable Token-Level Hallucination Detection in Large Language Models [63.3426544914783]
  Internal hallucinations are difficult to detect in reasoning-intensive tasks.<n>We propose TokenHD, a holistic pipeline for training token-level hallucination detectors.
  arXiv  Detail & Related papers  (2026-05-12T16:47:40Z)
• TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models [49.83690850047884]
  hallucination problem in D-LLMs remains underexplored, limiting their reliability in real-world applications.<n>Existing hallucination detection methods are designed for AR-LLMs and rely on signals from single-step generation.<n>We propose TraceDet, a novel framework that explicitly leverages the intermediate denoising steps of D-LLMs for hallucination detection.
  arXiv  Detail & Related papers  (2025-09-30T02:01:10Z)
• D$^2$HScore: Reasoning-Aware Hallucination Detection via Semantic Breadth and Depth Analysis in LLMs [15.665202830841046]
  This work revisits hallucination detection from the perspective of model architecture and generation dynamics.<n>We propose textbfD$2$HScore (Dispersion and Drift-based Hallucination Score), a training-free and label-free framework.<n>Experiments across five open-source Language Models and five widely used benchmarks demonstrate that D$2$HScore consistently outperforms existing training-free baselines.
  arXiv  Detail & Related papers  (2025-09-15T04:28:38Z)
• Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization [52.87635234206178]
  This paper proposes a new framework, namely MoNFAP, specifically tailored for multi-face manipulation detection and localization. The framework incorporates two novel modules: the Forgery-aware Unified Predictor (FUP) Module and the Mixture-of-Noises Module (MNM)
  arXiv  Detail & Related papers  (2024-08-05T08:35:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.