CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Models

• URL: http://arxiv.org/abs/2505.19108v1
• Date: Sun, 25 May 2025 11:54:32 GMT
• Title: CCHall: A Novel Benchmark for Joint Cross-Lingual and Cross-Modal Hallucinations Detection in Large Language Models
• Abstract summary: We introduce a novel joint Cross-lingual and Cross-modal Hallucinations benchmark (CCHall) to fill this gap.<n>Specifically, CCHall simultaneously incorporates both cross-lingual and cross-modal hallucination scenarios.<n>We conduct a comprehensive evaluation on CCHall, exploring both mainstream open-source and closed-source LLMs.
• Score: 20.692539273614763
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Investigating hallucination issues in large language models (LLMs) within cross-lingual and cross-modal scenarios can greatly advance the large-scale deployment in real-world applications. Nevertheless, the current studies are limited to a single scenario, either cross-lingual or cross-modal, leaving a gap in the exploration of hallucinations in the joint cross-lingual and cross-modal scenarios. Motivated by this, we introduce a novel joint Cross-lingual and Cross-modal Hallucinations benchmark (CCHall) to fill this gap. Specifically, CCHall simultaneously incorporates both cross-lingual and cross-modal hallucination scenarios, which can be used to assess the cross-lingual and cross-modal capabilities of LLMs. Furthermore, we conduct a comprehensive evaluation on CCHall, exploring both mainstream open-source and closed-source LLMs. The experimental results highlight that current LLMs still struggle with CCHall. We hope CCHall can serve as a valuable resource to assess LLMs in joint cross-lingual and cross-modal scenarios.

Related papers

• CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals? [3.390043185264311]
  Internal-state hallucination detection techniques have been explored predominantly in English.<n>We present CrossHallu, the first study to evaluate the cross-lingual and cross-domain generalization of hallucination detection.
  arXiv  Detail & Related papers  (2026-07-04T21:01:37Z)
• CCFQA: A Benchmark for Cross-Lingual and Cross-Modal Speech and Text Factuality Evaluation [26.054199546779696]
  CCFQA benchmark contains parallel speech-text factual questions across 8 languages.<n>Our experimental results demonstrate that current MLLMs still face substantial challenges on the CCFQA benchmark.<n>We propose a few-shot transfer learning strategy that effectively transfers the Question Answering (QA) capabilities of LLMs in English to multilingual Spoken Question Answering (SQA) tasks.
  arXiv  Detail & Related papers  (2025-08-10T11:09:41Z)
• CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation [23.610002725335313]
  Large Language Models (MLLMs) demonstrate strong generalization across languages, yet they remain prone to hallucinations, especially in low-resource languages.<n>We propose CCL-XCoT, a two-stage fine-tuning framework for mitigating hallucination in MLLMs.<n> Experimental results show that CCL-XCoT reduces hallucination rates by up to 62% and substantially improves factual knowledge transfer across language pairs.
  arXiv  Detail & Related papers  (2025-07-17T14:25:24Z)
• CLAIM: Mitigating Multilingual Object Hallucination in Large Vision-Language Models with Cross-Lingual Attention Intervention [56.08815340137881]
  Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal abilities but remain prone to multilingual object hallucination.<n>We propose Cross-Lingual Attention Intervention for Mitigating multilingual object hallucination (CLAIM) in LVLMs, a novel near training-free method by aligning attention patterns.
  arXiv  Detail & Related papers  (2025-06-03T11:17:16Z)
• Understanding LLMs' Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From [61.63091726904068]
  We evaluate the cross-lingual context retrieval ability of over 40 large language models (LLMs) across 12 languages.<n>Several small, post-trained open LLMs show strong cross-lingual context retrieval ability.<n>Our results also indicate that larger-scale pretraining cannot improve the xMRC performance.
  arXiv  Detail & Related papers  (2025-04-15T06:35:27Z)
• Exploring Cross-lingual Latent Transplantation: Mutual Opportunities and Open Challenges [48.96952594416528]
  Current large language models (LLMs) often exhibit imbalances in multilingual capabilities and cultural adaptability.<n>XTransplant framework enables models to harness the complementary strengths of both English and non-English resources by transplanting latent activations across languages.
  arXiv  Detail & Related papers  (2024-12-17T09:05:30Z)
• LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models [96.64960606650115]
  LongHalQA is an LLM-free hallucination benchmark that comprises 6K long and complex hallucination text. LongHalQA is featured by GPT4V-generated hallucinatory data that are well aligned with real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-13T18:59:58Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora.<n>But can these models relate corresponding concepts across languages, i.e., be crosslingual?<n>This study evaluates state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• Unveiling Linguistic Regions in Large Language Models [49.298360366468934]
  Large Language Models (LLMs) have demonstrated considerable cross-lingual alignment and generalization ability. This paper conducts several investigations on the linguistic competence of LLMs.
  arXiv  Detail & Related papers  (2024-02-22T16:56:13Z)
• Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance [2.371686365695081]
  Cross-lingual QA is a cross-lingual prompting method that translates only the question and answer parts, thus reducing translation costs. Experiments on four typologically diverse multilingual benchmarks show that Cross-lingual QA effectively stimulates models to elicit their cross-lingual knowledge. We show that prompting open-source MLLMs with cross-lingual in-context examples enhances performance as the model scale increases.
  arXiv  Detail & Related papers  (2023-05-24T15:14:49Z)
• Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment [24.742581572364124]
  In-context learning (ICL) unfolds as large language models become capable of inferring test labels conditioned on a few labeled samples without any gradient update. We provide the first in-depth analysis of ICL for cross-lingual text classification. We propose a novel prompt construction strategy -- Cross-lingual In-context Source-Target Alignment (X-InSTA)
  arXiv  Detail & Related papers  (2023-05-10T07:24:36Z)
• Zero-Shot Cross-Lingual Summarization via Large Language Models [108.30673793281987]
  Cross-lingual summarization ( CLS) generates a summary in a different target language. Recent emergence of Large Language Models (LLMs) has attracted wide attention from the computational linguistics community. In this report, we empirically use various prompts to guide LLMs to perform zero-shot CLS from different paradigms.
  arXiv  Detail & Related papers  (2023-02-28T01:27:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.