CLASH: A Benchmark for Cross-Modal Contradiction Detection

• URL: http://arxiv.org/abs/2511.19199v1
• Date: Mon, 24 Nov 2025 15:09:07 GMT
• Title: CLASH: A Benchmark for Cross-Modal Contradiction Detection
• Abstract summary: CLASH is a novel benchmark for multimodal contradiction detection.<n>It features COCO images paired with contradictory captions containing controlled object-level or attribute-level contradictions.
• Score: 15.134491772506196
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Contradictory multimodal inputs are common in real-world settings, yet existing benchmarks typically assume input consistency and fail to evaluate cross-modal contradiction detection - a fundamental capability for preventing hallucinations and ensuring reliability. We introduce CLASH, a novel benchmark for multimodal contradiction detection, featuring COCO images paired with contradictory captions containing controlled object-level or attribute-level contradictions. The samples include targeted questions evaluated in both multiple-choice and open-ended formats. The benchmark provides an extensive fine-tuning set filtered through automated quality checks, alongside a smaller human-verified diagnostic set. Our analysis of state-of-the-art models reveals substantial limitations in recognizing cross-modal conflicts, exposing systematic modality biases and category-specific weaknesses. Furthermore, we empirically demonstrate that targeted fine-tuning on CLASH substantially enhances conflict detection capabilities.

Related papers

• Quantifiable Uncertainty: A Stochastic Consensus Multi-Agent RAG Framework for Robust Malware Detection [1.2183405753834562]
  We present MAGMA, a framework that decouples malware analysis into semantic code retrieval and probabilistic verification.<n>We show that MAGMA achieves a 98.4% detection rate, substantially exceeding existing solutions.
  arXiv  Detail & Related papers  (2026-05-08T18:46:24Z)
• To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition [87.22988227382329]
  Multimodal emotion recognition (MER) benefits from combining text, audio, and vision, yet standard fusion often fails when modalities conflict.<n>We propose Dual-Path Conflict Resolution (DCR), a unified framework that learns when to fuse and when to drop modalities.
  arXiv  Detail & Related papers  (2026-05-06T13:11:33Z)
• CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs [43.70351207569419]
  We present CCTVBench, a Contrastive Consistency Traffic VideoQA Benchmark built on paired real accident videos and world-model-generated counterfactual counterparts.<n>CCTVBench enforces a single structured decision pattern over each video question quadruple and provides actionable diagnostics.<n>C-TCD, a contrastive decoding approach leveraging a semantically exclusive counterpart video as the contrast input at inference time, improves both instance-level QA and contrastive consistency.
  arXiv  Detail & Related papers  (2026-04-22T11:39:02Z)
• Beyond Nodes vs. Edges: A Multi-View Fusion Framework for Provenance-Based Intrusion Detection [6.343815622517112]
  PROVfusion is a multi-view anomaly fusion framework for provenance-based intrusion detection.<n>It fuses heterogeneous anomaly signals through lightweight fusion schemes and determines the final anomaly decisions through a voting-based integration process.<n>Experiments on nine widely used benchmark entity datasets demonstrate that PROVfusion achieves higher detection accuracy and lower false-positive rates than single node- and edge-centric baselines.
  arXiv  Detail & Related papers  (2026-04-16T06:40:50Z)
• Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection [105.14032334647932]
  Machine-generated texts (MGTs) pose risks such as disinformation and phishing, highlighting the need for reliable detection.<n> Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting.<n>We propose a Markov-informed score calibration strategy that models two relationships of context detection scores that may aid calibration.
  arXiv  Detail & Related papers  (2026-02-08T16:06:12Z)
• SetAD: Semi-Supervised Anomaly Learning in Contextual Sets [25.628827917857603]
  Semi-supervised anomaly detection has shown great promise by effectively leveraging limited labeled data.<n>We propose SetAD, a novel framework that reframes semi-supervised AD as a Set-level Anomaly Detection task.<n>To enhance robustness and score calibration, we propose a context-calibrated anomaly scoring mechanism.
  arXiv  Detail & Related papers  (2025-11-26T13:27:59Z)
• CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution [20.823419395675412]
  CrossCheck-Bench is a diagnostic benchmark for evaluating contradiction detection in multimodal inputs.<n>We evaluate 13 state-of-the-art vision-language models and observe a consistent performance drop as tasks shift from perceptual matching to logical contradiction detection.
  arXiv  Detail & Related papers  (2025-11-19T12:17:15Z)
• LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documents [0.10260880679794955]
  We present a multi-agent contradiction-aware benchmark framework for the legal domain.<n>It generates synthetic legal-style documents, injects six structured contradiction types, and models both self- and pairwise inconsistencies.<n>This benchmark offers one of the first structured resources for contradiction-aware evaluation in legal RAG pipelines.
  arXiv  Detail & Related papers  (2025-10-03T18:24:27Z)
• Unified Unsupervised Anomaly Detection via Matching Cost Filtering [113.43366521994396]
  Unsupervised anomaly detection (UAD) aims to identify image- and pixel-level anomalies using only normal training data.<n>We present Unified Cost Filtering (UCF), a generic post-hoc refinement framework for refining anomaly cost volume of any UAD model.
  arXiv  Detail & Related papers  (2025-10-03T03:28:18Z)
• METER: Multi-modal Evidence-based Thinking and Explainable Reasoning -- Algorithm and Benchmark [48.78602579128459]
  We introduce METER, a unified benchmark for interpretable forgery detection spanning images, videos, audio, and audio-visual content.<n>Our dataset comprises four tracks, each requiring not only real-vs-fake classification but also evidence-chain-based explanations.
  arXiv  Detail & Related papers  (2025-07-22T03:42:51Z)
• Fair Deepfake Detectors Can Generalize [51.21167546843708]
  We show that controlling for confounders (data distribution and model capacity) enables improved generalization via fairness interventions.<n>Motivated by this insight, we propose Demographic Attribute-insensitive Intervention Detection (DAID), a plug-and-play framework composed of: i) Demographic-aware data rebalancing, which employs inverse-propensity weighting and subgroup-wise feature normalization to neutralize distributional biases; and ii) Demographic-agnostic feature aggregation, which uses a novel alignment loss to suppress sensitive-attribute signals.<n>DAID consistently achieves superior performance in both fairness and generalization compared to several state-of-the-art
  arXiv  Detail & Related papers  (2025-07-03T14:10:02Z)
• CAD: A General Multimodal Framework for Video Deepfake Detection via Cross-Modal Alignment and Distillation [24.952907733127223]
  We propose a general framework for video deepfake detection via Cross-Modal Alignment and Distillation (CAD)<n>CAD comprises two core components: 1) Cross-modal alignment that identifies inconsistencies in high-level semantic synchronization (e.g., lip-speech mismatches); 2) Cross-modal distillation that mitigates mismatchs while preserving modality-specific forensic traces (e.g., spectral distortions in synthetic audio)
  arXiv  Detail & Related papers  (2025-05-21T08:11:07Z)
• Verify when Uncertain: Beyond Self-Consistency in Black Box Hallucination Detection [25.176984317213858]
  Large Language Models (LLMs) suffer from hallucination problems, which hinder their reliability in sensitive applications.<n>We propose a budget-friendly, two-stage detection algorithm that calls the verifier model only for a subset of cases.
  arXiv  Detail & Related papers  (2025-02-20T21:06:08Z)
• Addressing Key Challenges of Adversarial Attacks and Defenses in the Tabular Domain: A Methodological Framework for Coherence and Consistency [25.830427564563422]
  Class-Specific Anomaly Detection (CSAD) is an effective novel anomaly detection approach.<n> CSAD evaluates adversarial samples relative to their predicted class distribution, rather than a broad benign distribution.<n>Our evaluation incorporates both anomaly detection rates with SHAP-based assessments to provide a more comprehensive measure of adversarial sample quality.
  arXiv  Detail & Related papers  (2024-12-10T09:17:09Z)
• Evidential Deep Partial Multi-View Classification With Discount Fusion [24.139495744683128]
  We propose a novel framework called Evidential Deep Partial Multi-View Classification (EDP-MVC) We use K-means imputation to address missing views, creating a complete set of multi-view data. The potential conflicts and uncertainties within this imputed data can affect the reliability of downstream inferences.
  arXiv  Detail & Related papers  (2024-08-23T14:50:49Z)
• ADC: Adversarial attacks against object Detection that evade Context consistency checks [55.8459119462263]
  We show that even context consistency checks can be brittle to properly crafted adversarial examples. We propose an adaptive framework to generate examples that subvert such defenses. Our results suggest that how to robustly model context and check its consistency, is still an open problem.
  arXiv  Detail & Related papers  (2021-10-24T00:25:09Z)
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [74.05906222376608]
  We propose adversarial self-supervision UDA (or ASSUDA) that maximizes the agreement between clean images and their adversarial examples by a contrastive loss in the output space. This paper is rooted in two observations: (i) the robustness of UDA methods in semantic segmentation remains unexplored, which pose a security concern in this field; and (ii) although commonly used self-supervision (e.g., rotation and jigsaw) benefits image tasks such as classification and recognition, they fail to provide the critical supervision signals that could learn discriminative representation for segmentation tasks.
  arXiv  Detail & Related papers  (2021-05-23T01:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.