MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing

• URL: http://arxiv.org/abs/2605.24919v1
• Date: Sun, 24 May 2026 07:50:03 GMT
• Title: MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing
• Abstract summary: Hallucinations in Large Language Models (LLMs) represent a critical barrier to their reliable deployment.<n>We introduce MultiHaluDet, a novel three-stage stacking framework that detects multilingual hallucinations.<n>Our framework achieves state-of-the-art detection performance, reaching up to 98.55% AUROC on the English HaluEval and TriviaQA benchmarks.
• Score: 1.3700362496838856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hallucinations in Large Language Models (LLMs) represent a critical barrier to their reliable deployment, a vulnerability heavily exacerbated in non-English and resource-constrained contexts. Existing detection approaches that rely on output confidence heuristics or single-layer internal representations frequently fail to capture deep, complex factual inconsistencies across diverse languages. To address this, we introduce MultiHaluDet, a novel three-stage stacking framework that detects multilingual hallucinations by probing the full hidden state trajectories of frozen LLMs without requiring language-specific fine-tuning. Our method extracts sequential features across multiple layers and processes them via a hybrid architecture using multi-scale attention and self-attention pooling. By generating out-of-fold embeddings that feed into a calibrated classical classifier ensemble, MultiHaluDet captures both fine-grained and coarse-grained patterns of factual inconsistency. Extensive experiments demonstrate that our framework achieves state-of-the-art detection performance, reaching up to 98.55% AUROC on the English HaluEval and TriviaQA benchmarks using Mistral-7B and LLaMA2-7B architectures. Crucially, we rigorously evaluate our framework's cross-lingual generalization across high (French), medium (Bangla), and low-resource (Amharic) languages. MultiHaluDet demonstrates exceptional representational robustness, consistently outperforming baselines and successfully transferring hallucination detection capabilities across typologically diverse linguistic tiers.

Related papers

• All Languages Matter: Understanding and Mitigating Language Bias in Multilingual RAG [87.74635133954975]
  We show that current Multilingual Retrieval-Augmented Generation (mRAG) systems suffer from a language bias during reranking.<n>We propose textittextbfLanguage-textbfAgnostic textbfUtility-driven textbfReranker textbfAlignment (LAURA), which aligns multilingual evidence ranking with downstream generative utility.
  arXiv  Detail & Related papers  (2026-04-22T05:33:06Z)
• Layer-Targeted Multilingual Knowledge Erasure in Large Language Models [15.409568435026015]
  We identify intervention depth as the key factor determining multilingual generalization.<n>We propose MUTE, a framework that uses Centered Kernel Alignment (CKA) and Linguistic Regions Development Score (LRDS) to identify intermediate, language-agnostic layers.
  arXiv  Detail & Related papers  (2026-02-26T03:00:07Z)
• When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA [46.50540400870401]
  PsiloQA is a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages.<n>Our dataset and results advance the development of scalable, fine-grained hallucination detection in multilingual settings.
  arXiv  Detail & Related papers  (2025-10-06T14:36:30Z)
• Multilingual Routing in Mixture-of-Experts [45.90403983668531]
  We analyze expert routing patterns using parallel multilingual datasets.<n>We find that MoE models route tokens in language-specific ways in the early and late decoder layers.<n>We introduce a method that steers the router by promoting middle-layer task experts frequently activated in English.
  arXiv  Detail & Related papers  (2025-10-06T11:09:20Z)
• Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models [50.34755385896279]
  Confidence calibration is crucial for the reliable deployment of Large Language Models (LLMs)<n>We conduct the first large-scale, systematic studies of multilingual calibration across six model families and over 100 languages.<n>We find that non-English languages suffer from systematically worse calibration.
  arXiv  Detail & Related papers  (2025-10-03T16:07:15Z)
• Beyond ROUGE: N-Gram Subspace Features for LLM Hallucination Detection [5.0106565473767075]
  Large Language Models (LLMs) have demonstrated effectiveness across a wide variety of tasks involving natural language.<n>A fundamental problem of hallucinations still plagues these models, limiting their trustworthiness in generating consistent, truthful information.<n>We propose a novel approach inspired by ROUGE that constructs an N-Gram frequency tensor from LLM-generated text.<n>This tensor captures richer semantic structure by encoding co-occurrence patterns, enabling better differentiation between factual and hallucinated content.
  arXiv  Detail & Related papers  (2025-09-03T18:52:24Z)
• CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention [56.08815340137881]
  Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal abilities but remain prone to multilingual object hallucination.<n>We propose Cross-Lingual Attention Intervention for Mitigating multilingual object hallucination (CLAIM) in LVLMs, a novel near training-free method by aligning attention patterns.
  arXiv  Detail & Related papers  (2025-06-03T11:17:16Z)
• Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models [55.14276067678253]
  This paper introduces a novel methodology for efficiently identifying inherent cross-lingual weaknesses in Large Language Models (LLMs)<n>We construct a new dataset of over 6,000 bilingual pairs across 16 languages using this methodology, demonstrating its effectiveness in revealing weaknesses even in state-of-the-art models.<n>Further experiments investigate the relationship between linguistic similarity and cross-lingual weaknesses, revealing that linguistically related languages share similar performance patterns.
  arXiv  Detail & Related papers  (2025-05-24T12:31:27Z)
• MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety [56.77103365251923]
  Large Language Models (LLMs) are susceptible to adversarial attacks such as jailbreaking.<n>This vulnerability is exacerbated in multilingual settings, where multilingual safety-aligned data is often limited.<n>We introduce a multilingual guardrail with reasoning for prompt classification.
  arXiv  Detail & Related papers  (2025-04-21T17:15:06Z)
• Poly-FEVER: A Multilingual Fact Verification Benchmark for Hallucination Detection in Large Language Models [10.663446796160567]
  Hallucinations in generative AI, particularly in Large Language Models (LLMs), pose a significant challenge to the reliability of multilingual applications.<n>Existing benchmarks for hallucination detection focus primarily on English and a few widely spoken languages.<n>We introduce Poly-FEVER, a large-scale multilingual fact verification benchmark.
  arXiv  Detail & Related papers  (2025-03-19T01:46:09Z)
• Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval [51.60862829942932]
  We present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved. However, the peak performance is not met using the general-purpose multilingual text encoders off-the-shelf', but rather relying on their variants that have been further specialized for sentence understanding tasks.
  arXiv  Detail & Related papers  (2021-01-21T00:15:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.