CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?

• URL: http://arxiv.org/abs/2607.04029v1
• Date: Sat, 04 Jul 2026 21:01:37 GMT
• Title: CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?
• Abstract summary: Internal-state hallucination detection techniques have been explored predominantly in English.<n>We present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection.
• Score: 3.390043185264311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent hallucination detection techniques in large language models (LLMs) focus on directly extracting features from a model's internal representations and training a classifier on these features to detect hallucinations, demonstrating promising results. Notwithstanding this advancement, most internal-state hallucination detection techniques have been explored predominantly in English, raising the question of whether such internal signals generalize across different languages and domains. To address this gap, we present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection using internal representations from six LLMs on the generative question-answering task. We conduct a systematic Arabic <-> English evaluation using TruthfulQA, an Arabic translated version of TruthfulQA, and HalluScore. This evaluation encompasses monolingual training and testing, cross-lingual transfer, cross-domain transfer, and combined cross-lingual and cross-domain transfer. The results reveal that internal-state hallucination signals in LLMs transfer across languages and domains for most models, with cross-lingual performance highly dependent on both class separability and language alignment in the feature space, whereas cross-domain transfer within Arabic varies depending on the training and testing datasets used for the hallucination detector. The code is publicly available at https://github.com/aishaalansari57/CrossHal.

Related papers

• When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training [57.230355403478995]
  We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM.<n>We find that shared concept spaces emerge early and continue to refine, but that alignment with them is language-dependent.<n>In contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior.
  arXiv  Detail & Related papers  (2026-01-30T11:23:01Z)
• Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders [51.380449540006985]
  Large Language Models (LLMs) can process many languages, yet how they internally represent this diversity remains unclear.<n>Do they form shared multilingual representations with language-specific decoding, and if so, why does performance still favor the dominant training language?<n>We analyze their internal mechanisms using cross-layer transcoders (CLT) and attribution graphs.
  arXiv  Detail & Related papers  (2025-11-13T22:51:06Z)
• LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs [67.09110757873142]
  We present LiveCLKTBench, an automated generation pipeline designed to isolate and measure cross-lingual knowledge transfer.<n>Our pipeline identifies self-contained, time-sensitive knowledge entities from real-world domains.<n>The documents of these valid entities are then used to generate factual questions, which are translated into multiple languages.
  arXiv  Detail & Related papers  (2025-11-03T17:06:49Z)
• Ask a Local: Detecting Hallucinations With Specialized Model Divergence [0.16874375111244325]
  We introduce "Ask a Local", a novel hallucination detection method for large language models.<n>Our approach computes divergence between perplexity distributions of language-specialized models to identify potentially hallucinated spans.<n>Our results on a human-annotated question-answer dataset spanning 14 languages demonstrate consistent performance across languages.
  arXiv  Detail & Related papers  (2025-06-03T20:00:49Z)
• CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention [56.08815340137881]
  Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal abilities but remain prone to multilingual object hallucination.<n>We propose Cross-Lingual Attention Intervention for Mitigating multilingual object hallucination (CLAIM) in LVLMs, a novel near training-free method by aligning attention patterns.
  arXiv  Detail & Related papers  (2025-06-03T11:17:16Z)
• CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Models [20.692539273614763]
  We introduce a novel joint Cross-lingual and Cross-modal Hallucinations benchmark (CCHall) to fill this gap.<n>Specifically, CCHall simultaneously incorporates both cross-lingual and cross-modal hallucination scenarios.<n>We conduct a comprehensive evaluation on CCHall, exploring both mainstream open-source and closed-source LLMs.
  arXiv  Detail & Related papers  (2025-05-25T11:54:32Z)
• ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection [3.1269598124014264]
  We introduce ANHALTEN, a dataset that extends the English hallucination detection dataset to German. This is the first work that explores cross-lingual transfer for token-level reference-free hallucination detection. We show that the sample-efficient few-shot transfer is the most effective approach in most setups.
  arXiv  Detail & Related papers  (2024-07-18T17:01:38Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora.<n>But can these models relate corresponding concepts across languages, i.e., be crosslingual?<n>This study evaluates state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• Probing the Emergence of Cross-lingual Alignment during LLM Training [10.053333786023089]
  Multilingual Large Language Models (LLMs) achieve remarkable levels of zero-shot cross-lingual transfer performance. We study how such cross-lingual alignment emerges during pre-training of LLMs. We observe a high correlation between neuron overlap and downstream performance.
  arXiv  Detail & Related papers  (2024-06-19T05:31:59Z)
• Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models [73.11488464916668]
  This study investigates the dynamics of the multilingual pretraining process. We probe checkpoints taken from throughout XLM-R pretraining, using a suite of linguistic tasks. Our analysis shows that the model achieves high in-language performance early on, with lower-level linguistic skills acquired before more complex ones.
  arXiv  Detail & Related papers  (2022-05-24T03:35:00Z)
• Analyzing Zero-shot Cross-lingual Transfer in Supervised NLP Tasks [6.7155846430379285]
  In zero-shot cross-lingual transfer, a supervised NLP task trained on a corpus in one language is directly applicable to another language without any additional training. Recently introduced cross-lingual language model (XLM) pretraining brings out neural parameter sharing in Transformer-style networks. In this paper, we aim to validate the hypothetically strong cross-lingual transfer properties induced by XLM pretraining.
  arXiv  Detail & Related papers  (2021-01-26T09:21:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.