Modality Interactive Mixture-of-Experts for Fake News Detection

• URL: http://arxiv.org/abs/2501.12431v1
• Date: Tue, 21 Jan 2025 16:49:00 GMT
• Title: Modality Interactive Mixture-of-Experts for Fake News Detection
• Authors: Yifan Liu, Yaokun Liu, Zelin Li, Ruichen Yao, Yang Zhang, Dong Wang,
• Abstract summary: We present Modality Interactive Mixture-of-Experts for Fake News Detection (MIMoE-FND) MIMoE-FND is a novel hierarchical Mixture-of-Experts framework designed to enhance multimodal fake news detection. We evaluate our approach on three real-world benchmarks spanning two languages, demonstrating its superior performance compared to state-of-the-art methods.
• Score: 13.508494216511094
• License:
• Abstract: The proliferation of fake news on social media platforms disproportionately impacts vulnerable populations, eroding trust, exacerbating inequality, and amplifying harmful narratives. Detecting fake news in multimodal contexts -- where deceptive content combines text and images -- is particularly challenging due to the nuanced interplay between modalities. Existing multimodal fake news detection methods often emphasize cross-modal consistency but ignore the complex interactions between text and visual elements, which may complement, contradict, or independently influence the predicted veracity of a post. To address these challenges, we present Modality Interactive Mixture-of-Experts for Fake News Detection (MIMoE-FND), a novel hierarchical Mixture-of-Experts framework designed to enhance multimodal fake news detection by explicitly modeling modality interactions through an interaction gating mechanism. Our approach models modality interactions by evaluating two key aspects of modality interactions: unimodal prediction agreement and semantic alignment. The hierarchical structure of MIMoE-FND allows for distinct learning pathways tailored to different fusion scenarios, adapting to the unique characteristics of each modality interaction. By tailoring fusion strategies to diverse modality interaction scenarios, MIMoE-FND provides a more robust and nuanced approach to multimodal fake news detection. We evaluate our approach on three real-world benchmarks spanning two languages, demonstrating its superior performance compared to state-of-the-art methods. By enhancing the accuracy and interpretability of fake news detection, MIMoE-FND offers a promising tool to mitigate the spread of misinformation, with the potential to better safeguard vulnerable communities against its harmful effects.

Related papers

• Multimodal Inverse Attention Network with Intrinsic Discriminant Feature Exploitation for Fake News Detection [15.472199961982271]
  Multimodal fake news detection has garnered significant attention due to its profound implications for social security. We propose a novel framework that explores intrinsic discriminative features based on news content to advance fake news detection.
  arXiv  Detail & Related papers  (2025-02-03T07:58:22Z)
• RaCMC: Residual-Aware Compensation Network with Multi-Granularity Constraints for Fake News Detection [24.91068046428761]
  Multimodal fake news detection aims to automatically identify real or fake news, thereby mitigating the adverse effects caused by such misinformation. We present a residual-aware compensation network with multi-granularity constraints for fake news detection, that aims to sufficiently interact and fuse cross-modal features. Experiments on three public datasets, including Weibo17, Politifact and GossipCop, reveal the superiority of the proposed method.
  arXiv  Detail & Related papers  (2024-12-24T08:08:29Z)
• MSynFD: Multi-hop Syntax aware Fake News Detection [27.046529059563863]
  Social media platforms have fueled the rapid dissemination of fake news, posing threats to our real-life society. Existing methods use multimodal data or contextual information to enhance the detection of fake news. We propose a novel multi-hop syntax aware fake news detection (MSynFD) method, which incorporates complementary syntax information to deal with subtle twists in fake news.
  arXiv  Detail & Related papers  (2024-02-18T05:40:33Z)
• From Text to Pixels: A Context-Aware Semantic Synergy Solution for Infrared and Visible Image Fusion [66.33467192279514]
  We introduce a text-guided multi-modality image fusion method that leverages the high-level semantics from textual descriptions to integrate semantics from infrared and visible images. Our method not only produces visually superior fusion results but also achieves a higher detection mAP over existing methods, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2023-12-31T08:13:47Z)
• MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts [92.76662894585809]
  We introduce an approach to enhance multimodal models, which we call Multimodal Mixtures of Experts (MMoE) MMoE is able to be applied to various types of models to gain improvement.
  arXiv  Detail & Related papers  (2023-11-16T05:31:21Z)
• Detecting and Grounding Multi-Modal Media Manipulation and Beyond [93.08116982163804]
  We highlight a new research problem for multi-modal fake media, namely Detecting and Grounding Multi-Modal Media Manipulation (DGM4) DGM4 aims to not only detect the authenticity of multi-modal media, but also ground the manipulated content. We propose a novel HierArchical Multi-modal Manipulation rEasoning tRansformer (HAMMER) to fully capture the fine-grained interaction between different modalities.
  arXiv  Detail & Related papers  (2023-09-25T15:05:46Z)
• Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detection [57.13665112065285]
  Human-Object Interaction (HOI) detection is a challenging computer vision task. We present a framework that enhances HOI detection by incorporating structured text knowledge.
  arXiv  Detail & Related papers  (2023-07-25T14:20:52Z)
• Multi-Grained Multimodal Interaction Network for Entity Linking [65.30260033700338]
  Multimodal entity linking task aims at resolving ambiguous mentions to a multimodal knowledge graph. We propose a novel Multi-GraIned Multimodal InteraCtion Network $textbf(MIMIC)$ framework for solving the MEL task.
  arXiv  Detail & Related papers  (2023-07-19T02:11:19Z)
• Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusion [21.042970740577648]
  We present a Multi-grained Multi-modal Fusion Network (MMFN) for fake news detection. Inspired by the multi-grained process of human assessment of news authenticity, we respectively employ two Transformer-based pre-trained models to encode token-level features from text and images. The multi-modal module fuses fine-grained features, taking into account coarse-grained features encoded by the CLIP encoder.
  arXiv  Detail & Related papers  (2023-04-03T09:13:59Z)
• Cross-modal Contrastive Learning for Multimodal Fake News Detection [10.760000041969139]
  COOLANT is a cross-modal contrastive learning framework for multimodal fake news detection. A cross-modal fusion module is developed to learn the cross-modality correlations. An attention guidance module is implemented to help effectively and interpretably aggregate the aligned unimodal representations.
  arXiv  Detail & Related papers  (2023-02-25T10:12:34Z)
• Multimodal Categorization of Crisis Events in Social Media [81.07061295887172]
  We present a new multimodal fusion method that leverages both images and texts as input. In particular, we introduce a cross-attention module that can filter uninformative and misleading components from weak modalities. We show that our method outperforms the unimodal approaches and strong multimodal baselines by a large margin on three crisis-related tasks.
  arXiv  Detail & Related papers  (2020-04-10T06:31:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.