Tri-PvP: Exposing Modality Bias in Omni-Modal Large Language Models through Perceptual-Propositional Evidence Conflicts

• URL: http://arxiv.org/abs/2609.06011v1
• Date: Sat, 05 Sep 2026 10:33:58 GMT
• Title: Tri-PvP: Exposing Modality Bias in Omni-Modal Large Language Models through Perceptual-Propositional Evidence Conflicts
• Abstract summary: Existing benchmarks conflate two forms of evidence within a single modality: perceptual signals and propositional signals.<n>We introduce Tri-PvP, a benchmark crossing vision, audio, and text, where vision and audio each take perceptual or propositional form.<n>We find robust visual bias across most models and evidence-type conditions. Crucially, we reveal a systematic asymmetry in evidence-form bias.
• Score: 49.97842835825998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Omni-modal large language models (OLLMs) jointly process vision, audio, and text, yet their modality bias under cross-modal conflict remains underexplored. Existing benchmarks conflate two distinct forms of evidence within a single modality: perceptual signals (e.g., a photograph or recording of a dog) and propositional signals (e.g., the declarative claim "this is a dog"), such that any measured modality bias is inherently confounded with evidence-form bias, precluding clean attribution to either source. To address this, we introduce Tri-PvP, an 8,000-sample tri-modal conflict benchmark crossing vision, audio, and text, where vision and audio each take perceptual or propositional form. Evaluating five OLLMs, we find robust visual bias across most models and evidence-type conditions. Crucially, we reveal a systematic asymmetry in evidence-form bias: models exhibit a stronger bias toward perceptual signal in vision but propositional in audio. Further analyses via layer-wise linear probing and contrastive decoding reveal that modality bias is already linearly decodable from early representation layers and can only be partially mitigated, calling for mitigation strategies beyond surface-level interventions.

Related papers

• Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation [41.3823869123186]
  Multimodal Large Language Models (MLLMs) are prone to hallucination as their generation preferences are insufficiently calibrated to visual evidence.<n>We propose Oriented Pickup Preference Optimization (textttOPPO), an evidence-aware alignment objective that learns preferences over the strength of visual evidence.
  arXiv  Detail & Related papers  (2026-06-29T05:33:22Z)
• Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought [49.53567098922619]
  We propose Separate First, Fuse Later (SFFL) as an audio-visual reasoning framework to reduce cross-modal interference.<n>SFFL enforces modality-specific chain-of-thought reasoning, producing separate audio and visual reasoning traces and integrating evidence for answering.<n> Experiments demonstrate consistent improvements in both accuracy and robustness, yielding an average relative gain of 5.16% on general AVQA benchmarks and 11.17% on a cross-modal hallucination benchmark.
  arXiv  Detail & Related papers  (2026-05-11T02:50:27Z)
• PRIMED: Adaptive Modality Suppression for Referring Audio-Visual Segmentation via Biased Competition [9.145417637255651]
  Referring Audio-Visual (Ref-AVS) seeks to localize and segment target objects in video frames based on visual, auditory, and textual referring cues.<n>We propose PRIMED, inspired by the biased competition theory in cognitive neuroscience, which explicitly models both visual perception and language-driven prior modulation.
  arXiv  Detail & Related papers  (2026-05-08T02:40:34Z)
• Integrating Fine-Grained Audio-Visual Evidence for Robust Multimodal Emotion Reasoning [9.470507126417292]
  We introduce SABER-LLM, a framework designed for robust multimodal reasoning.<n>First, we construct SABER, a large-scale emotion reasoning dataset comprising 600K video clips.<n>Second, we propose the structured evidence decomposition paradigm, which enforces a "perceive-then-reason" separation between evidence extraction and reasoning.
  arXiv  Detail & Related papers  (2026-01-26T10:03:26Z)
• DIVER: Dynamic Iterative Visual Evidence Reasoning for Multimodal Fake News Detection [6.225860651499494]
  Multimodal fake news detection is crucial for mitigating adversarial misinformation.<n>We propose DIVER (Dynamic Iterative Visual Evidence Reasoning), a framework grounded in a progressive, evidence-driven reasoning paradigm.<n>Experiments on Weibo, Weibo21, and GossipCop demonstrate that DIVER outperforms state-of-the-art baselines by an average of 2.72%.
  arXiv  Detail & Related papers  (2026-01-12T04:01:33Z)
• Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts [74.47786985522762]
  We identify a critical failure mode termed textual inertia, where models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence.<n>We propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the reasoning chains of diverse LMMs.<n>Results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation.
  arXiv  Detail & Related papers  (2026-01-07T16:39:34Z)
• Towards Unification of Hallucination Detection and Fact Verification for Large Language Models [31.407333541614296]
  Large Language Models (LLMs) frequently exhibit hallucinations, generating content that appears fluent but is factually incorrect.<n>To address this challenge, two distinct research paradigms have emerged: model-centric Hallucination Detection (HD) and text-centric Fact Verification (FV)<n>We introduce UniFact, a unified evaluation framework that enables direct, instance-level comparison between FV and HD.
  arXiv  Detail & Related papers  (2025-12-02T13:51:01Z)
• RoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulation [67.38036090822982]
  We propose RoboView-Bias, the first benchmark specifically designed to quantify visual bias in robotic manipulation.<n>We create 2,127 task instances that enable robust measurement of biases induced by individual visual factors and their interactions.<n>Our results highlight that systematic analysis of visual bias is a prerequisite for developing safe and reliable general-purpose embodied agents.
  arXiv  Detail & Related papers  (2025-09-26T13:53:25Z)
• A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models [58.32070787537946]
  Chain-of-thought (CoT) reasoning enhances performance of large language models.<n>We present the first comprehensive study of CoT faithfulness in large vision-language models.
  arXiv  Detail & Related papers  (2025-05-29T18:55:05Z)
• Debiasing Multimodal Large Language Models via Penalization of Language Priors [38.97645845493758]
  Multimodal Large Language Models (MLLMs) have become indispensable tools in computer vision and natural language processing.<n>Despite their advancements, our investigation reveals a noteworthy bias: the generated content is often driven more by the inherent priors of the underlying Large Language Models (LLMs) than by the input image.<n>We propose two simple, training-free strategies to rectify these biases and redirect the model's focus toward visual information.
  arXiv  Detail & Related papers  (2024-03-08T12:35:07Z)
• A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition [53.800937914403654]
  Advanced Audio-Visual Speech Recognition (AVSR) systems have been observed to be sensitive to missing video frames. While applying the dropout technique to the video modality enhances robustness to missing frames, it simultaneously results in a performance loss when dealing with complete data input. We propose a novel Multimodal Distribution Approximation with Knowledge Distillation (MDA-KD) framework to reduce over-reliance on the audio modality.
  arXiv  Detail & Related papers  (2024-03-07T06:06:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.