Deconfounded Reasoning for Multimodal Fake News Detection via Causal Intervention

• URL: http://arxiv.org/abs/2504.09163v1
• Date: Sat, 12 Apr 2025 09:57:43 GMT
• Title: Deconfounded Reasoning for Multimodal Fake News Detection via Causal Intervention
• Authors: Moyang Liu, Kaiying Yan, Yukun Liu, Ruibo Fu, Zhengqi Wen, Xuefei Liu, Chenxing Li,
• Abstract summary: The rapid growth of social media has led to the widespread dissemination of fake news across multiple content forms.<n>Traditional unimodal detection methods fall short in addressing complex cross-modal manipulations.<n>We propose the Causal Intervention-based Multimodal Decon Detection framework.
• Score: 16.607714608483164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The rapid growth of social media has led to the widespread dissemination of fake news across multiple content forms, including text, images, audio, and video. Traditional unimodal detection methods fall short in addressing complex cross-modal manipulations; as a result, multimodal fake news detection has emerged as a more effective solution. However, existing multimodal approaches, especially in the context of fake news detection on social media, often overlook the confounders hidden within complex cross-modal interactions, leading models to rely on spurious statistical correlations rather than genuine causal mechanisms. In this paper, we propose the Causal Intervention-based Multimodal Deconfounded Detection (CIMDD) framework, which systematically models three types of confounders via a unified Structural Causal Model (SCM): (1) Lexical Semantic Confounder (LSC); (2) Latent Visual Confounder (LVC); (3) Dynamic Cross-Modal Coupling Confounder (DCCC). To mitigate the influence of these confounders, we specifically design three causal modules based on backdoor adjustment, frontdoor adjustment, and cross-modal joint intervention to block spurious correlations from different perspectives and achieve causal disentanglement of representations for deconfounded reasoning. Experimental results on the FakeSV and FVC datasets demonstrate that CIMDD significantly improves detection accuracy, outperforming state-of-the-art methods by 4.27% and 4.80%, respectively. Furthermore, extensive experimental results indicate that CIMDD exhibits strong generalization and robustness across diverse multimodal scenarios.

Related papers

• Consistency-aware Fake Videos Detection on Short Video Platforms [4.291448222735821]
  This paper focuses on detecting fake news on the short video platforms. Existing approaches typically combine raw video data with metadata inputs before applying a classification layer. Motivated by this insight, we propose a novel detection paradigm that explicitly identifies and leverages cross-modal contradictions.
  arXiv  Detail & Related papers  (2025-04-30T10:26:04Z)
• Exploring Modality Disruption in Multimodal Fake News Detection [16.607714608483164]
  We propose a multimodal fake news detection framework, FND-MoE, to address the issue of modality disruption.<n>FND-MoE significantly outperforms state-of-the-art methods, with accuracy improvements of 3.45% and 3.71% on the respective datasets.
  arXiv  Detail & Related papers  (2025-04-12T09:39:29Z)
• Modality Interactive Mixture-of-Experts for Fake News Detection [13.508494216511094]
  We present Modality Interactive Mixture-of-Experts for Fake News Detection (MIMoE-FND)<n>MIMoE-FND is a novel hierarchical Mixture-of-Experts framework designed to enhance multimodal fake news detection.<n>We evaluate our approach on three real-world benchmarks spanning two languages, demonstrating its superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-21T16:49:00Z)
• Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understanding [62.70450216120704]
  Unsupervised pre-training has shown great success in skeleton-based action understanding. We propose a Unified Multimodal Unsupervised Representation Learning framework, called UmURL. UmURL exploits an efficient early-fusion strategy to jointly encode the multi-modal features in a single-stream manner.
  arXiv  Detail & Related papers  (2023-11-06T13:56:57Z)
• Detecting and Grounding Multi-Modal Media Manipulation and Beyond [93.08116982163804]
  We highlight a new research problem for multi-modal fake media, namely Detecting and Grounding Multi-Modal Media Manipulation (DGM4) DGM4 aims to not only detect the authenticity of multi-modal media, but also ground the manipulated content. We propose a novel HierArchical Multi-modal Manipulation rEasoning tRansformer (HAMMER) to fully capture the fine-grained interaction between different modalities.
  arXiv  Detail & Related papers  (2023-09-25T15:05:46Z)
• Multi-Grained Multimodal Interaction Network for Entity Linking [65.30260033700338]
  Multimodal entity linking task aims at resolving ambiguous mentions to a multimodal knowledge graph. We propose a novel Multi-GraIned Multimodal InteraCtion Network $textbf(MIMIC)$ framework for solving the MEL task.
  arXiv  Detail & Related papers  (2023-07-19T02:11:19Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusion [21.042970740577648]
  We present a Multi-grained Multi-modal Fusion Network (MMFN) for fake news detection. Inspired by the multi-grained process of human assessment of news authenticity, we respectively employ two Transformer-based pre-trained models to encode token-level features from text and images. The multi-modal module fuses fine-grained features, taking into account coarse-grained features encoded by the CLIP encoder.
  arXiv  Detail & Related papers  (2023-04-03T09:13:59Z)
• Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework [89.8609061423685]
  We propose an information-theoretic approach to quantify the degree of redundancy, uniqueness, and synergy relating input modalities with an output task. To validate PID estimation, we conduct extensive experiments on both synthetic datasets where the PID is known and on large-scale multimodal benchmarks. We demonstrate their usefulness in (1) quantifying interactions within multimodal datasets, (2) quantifying interactions captured by multimodal models, (3) principled approaches for model selection, and (4) three real-world case studies.
  arXiv  Detail & Related papers  (2023-02-23T18:59:05Z)
• Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis [96.46952672172021]
  Bi-Bimodal Fusion Network (BBFN) is a novel end-to-end network that performs fusion on pairwise modality representations. Model takes two bimodal pairs as input due to known information imbalance among modalities.
  arXiv  Detail & Related papers  (2021-07-28T23:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.