OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

• URL: http://arxiv.org/abs/2609.11244v1
• Date: Thu, 10 Sep 2026 08:41:12 GMT
• Title: OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models
• Abstract summary: We introduce OmniHallu, a unified hallucination detection framework spanning both comprehension and generation tasks.<n>Our multi-agent architecture decomposes model outputs into atomic claims, verifies them through modality-specific experts, and aggregates evidence via structured reasoning.
• Score: 2.9934878496863946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse tasks, they suffer from hallucinations where generated outputs contradict or misrepresent input semantics. Existing research typically addresses hallucination detection within a single modality or task type, limiting generalizability. We introduce OmniHallu, a unified hallucination detection framework spanning both comprehension and generation tasks across image, video, and audio modalities. We contribute OmniHallu-Bench, a 10,000-sample benchmark with claim-level human annotations covering six cross-modal tasks: image-to-text (I2T), video-to-text (V2T), audio-to-text (A2T), text-to-image (T2I), text-to-video (T2V), and text-to-audio (T2A). Our multi-agent architecture decomposes model outputs into atomic claims, verifies them through modality-specific experts, and aggregates evidence via structured reasoning. We further propose a preference-optimized trainable verifier that approximates the multi-agent decision boundary, reducing expert calls by 66% with minimal performance loss. Extensive experiments reveal a consistent modality-dependent performance gradient and provide fine-grained insights into cross-modal hallucination patterns.

Related papers

• UniProbe: A Learnable Token-Level Hallucination Detector for Large VLMs using Multi-Structural Internal Representations [61.85916280854257]
  Large Vision-Language Models (LVLMs) achieve impressive visual reasoning and dialogue capabilities, yet frequently hallucinate content unsupported by the visual input.<n>We introduce textbfUniProbe, a lightweight, unified, learnable detector that models a frozen LVLM's heterogeneous computational trace from a single forward pass.
  arXiv  Detail & Related papers  (2026-08-11T12:01:59Z)
• MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing [1.3700362496838856]
  Hallucinations in Large Language Models (LLMs) represent a critical barrier to their reliable deployment.<n>We introduce MultiHaluDet, a novel three-stage stacking framework that detects multilingual hallucinations.<n>Our framework achieves state-of-the-art detection performance, reaching up to 98.55% AUROC on the English HaluEval and TriviaQA benchmarks.
  arXiv  Detail & Related papers  (2026-05-24T07:50:03Z)
• OmniRet: Efficient and High-Fidelity Omni Modality Retrieval [51.80205678389465]
  We present OmniRet, the first retrieval model capable of handling complex, composed queries spanning three key modalities: text, vision, and audio.<n>Our model demonstrates significant improvements on composed query, audio and video retrieval tasks, while achieving on-par performance with state-of-the-art models on others.
  arXiv  Detail & Related papers  (2026-03-02T17:19:55Z)
• Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection [6.491407316650203]
  Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) have shown strong generalisation in detecting image and video deepfakes.<n>We aim to explore the potential of MLLMs for audio deepfake detection.
  arXiv  Detail & Related papers  (2026-01-02T18:17:22Z)
• Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception [97.32606786622728]
  We present a systematic and comprehensive investigation of omni detailed perception from the perspectives of the data pipeline, models, and benchmark.<n>We propose Omni-Detective, an agentic data generation pipeline integrating tool-calling, to autonomously produce highly detailed yet minimally hallucinatory multimodal data.<n>Based on the data generated with Omni-Detective, we train two captioning models: Audio-Captioner for audio-only detailed perception, and Omni-Captioner for audio-visual detailed perception.
  arXiv  Detail & Related papers  (2025-10-14T17:00:09Z)
• Beyond ROUGE: N-Gram Subspace Features for LLM Hallucination Detection [5.0106565473767075]
  Large Language Models (LLMs) have demonstrated effectiveness across a wide variety of tasks involving natural language.<n>A fundamental problem of hallucinations still plagues these models, limiting their trustworthiness in generating consistent, truthful information.<n>We propose a novel approach inspired by ROUGE that constructs an N-Gram frequency tensor from LLM-generated text.<n>This tensor captures richer semantic structure by encoding co-occurrence patterns, enabling better differentiation between factual and hallucinated content.
  arXiv  Detail & Related papers  (2025-09-03T18:52:24Z)
• OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination [32.43796002503023]
  We propose OmniDPO, a preference-alignment framework designed to mitigate hallucinations in Omni-modal large language models (OLLMs)<n>By tackling both challenges, OmniDPO effectively improves multimodal grounding and reduces hallucination.
  arXiv  Detail & Related papers  (2025-08-31T07:19:32Z)
• A Survey of Multimodal Hallucination Evaluation and Detection [52.03164192840023]
  Multi-modal Large Language Models (MLLMs) have emerged as a powerful paradigm for integrating visual and textual information.<n>These models often suffer from hallucination, producing content that appears plausible but contradicts the input content or established world knowledge.<n>This survey offers an in-depth review of hallucination evaluation benchmarks and detection methods across Image-to-Text (I2T) and Text-to-image (T2I) generation tasks.
  arXiv  Detail & Related papers  (2025-07-25T07:22:42Z)
• M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation [24.480587619037184]
  Video-grounded dialogue generation (VDG) requires the system to generate a fluent and accurate answer based on multimodal knowledge. The difficulty in multimodal knowledge utilization brings serious hallucinations to VDG models in practice. We propose M2K-VDG, a model-adaptive multimodal knowledge anchor enhancement framework for hallucination reduction.
  arXiv  Detail & Related papers  (2024-02-19T06:32:39Z)
• AutoHall: Automated Hallucination Dataset Generation for Large Language Models [56.92068213969036]
  This paper introduces a method for automatically constructing model-specific hallucination datasets based on existing fact-checking datasets called AutoHall. We also propose a zero-resource and black-box hallucination detection method based on self-contradiction.
  arXiv  Detail & Related papers  (2023-09-30T05:20:02Z)
• MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text [58.655375327681774]
  We propose the first Multimodal Retrieval-Augmented Transformer (MuRAG) MuRAG accesses an external non-parametric multimodal memory to augment language generation. Our results show that MuRAG achieves state-of-the-art accuracy, outperforming existing models by 10-20% absolute on both datasets.
  arXiv  Detail & Related papers  (2022-10-06T13:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.