Robust Domain Misinformation Detection via Multi-modal Feature Alignment
- URL: http://arxiv.org/abs/2311.14315v1
- Date: Fri, 24 Nov 2023 07:06:16 GMT
- Title: Robust Domain Misinformation Detection via Multi-modal Feature Alignment
- Authors: Hui Liu, Wenya Wang, Hao Sun, Anderson Rocha, and Haoliang Li
- Abstract summary: We propose a robust domain and cross-modal approach for multi-modal misinformation detection.
It reduces the domain shift by aligning the joint distribution of textual and visual modalities.
We also propose a framework that simultaneously considers application scenarios of domain generalization.
- Score: 49.89164555394584
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Social media misinformation harms individuals and societies and is
potentialized by fast-growing multi-modal content (i.e., texts and images),
which accounts for higher "credibility" than text-only news pieces. Although
existing supervised misinformation detection methods have obtained acceptable
performances in key setups, they may require large amounts of labeled data from
various events, which can be time-consuming and tedious. In turn, directly
training a model by leveraging a publicly available dataset may fail to
generalize due to domain shifts between the training data (a.k.a. source
domains) and the data from target domains. Most prior work on domain shift
focuses on a single modality (e.g., text modality) and ignores the scenario
where sufficient unlabeled target domain data may not be readily available in
an early stage. The lack of data often happens due to the dynamic propagation
trend (i.e., the number of posts related to fake news increases slowly before
catching the public attention). We propose a novel robust domain and
cross-modal approach (\textbf{RDCM}) for multi-modal misinformation detection.
It reduces the domain shift by aligning the joint distribution of textual and
visual modalities through an inter-domain alignment module and bridges the
semantic gap between both modalities through a cross-modality alignment module.
We also propose a framework that simultaneously considers application scenarios
of domain generalization (in which the target domain data is unavailable) and
domain adaptation (in which unlabeled target domain data is available).
Evaluation results on two public multi-modal misinformation detection datasets
(Pheme and Twitter Datasets) evince the superiority of the proposed model. The
formal implementation of this paper can be found in this link:
https://github.com/less-and-less-bugs/RDCM
Related papers
- DPOD: Domain-Specific Prompt Tuning for Multimodal Fake News Detection [15.599951180606947]
Fake news using out-of-context images has become widespread and is a relevant problem in this era of information overload.
We explore whether out-of-domain data can help to improve out-of-context misinformation detection of a desired domain.
We propose a novel framework termed DPOD (Domain-specific Prompt-tuning using Out-of-Domain data)
arXiv Detail & Related papers (2023-11-27T08:49:26Z) - Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval [55.122020263319634]
Video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query.
In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain but the domain of interest only contains unannotated datasets.
We propose a novel Multi-Modal Cross-Domain Alignment network to transfer the annotation knowledge from the source domain to the target domain.
arXiv Detail & Related papers (2022-09-23T12:58:20Z) - Multilevel Knowledge Transfer for Cross-Domain Object Detection [26.105283273950942]
Domain shift is a well known problem where a model trained on a particular domain (source) does not perform well when exposed to samples from a different domain (target)
In this work, we address the domain shift problem for the object detection task.
Our approach relies on gradually removing the domain shift between the source and the target domains.
arXiv Detail & Related papers (2021-08-02T15:24:40Z) - Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available.
This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets.
We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
arXiv Detail & Related papers (2021-03-25T14:33:33Z) - A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [81.07994783143533]
Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks.
In many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data.
To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
arXiv Detail & Related papers (2020-09-01T00:06:50Z) - Domain Adaptation for Semantic Parsing [68.81787666086554]
We propose a novel semantic for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain.
Our semantic benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages.
Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies.
arXiv Detail & Related papers (2020-06-23T14:47:41Z) - Mind the Gap: Enlarging the Domain Gap in Open Set Domain Adaptation [65.38975706997088]
Open set domain adaptation (OSDA) assumes the presence of unknown classes in the target domain.
We show that existing state-of-the-art methods suffer a considerable performance drop in the presence of larger domain gaps.
We propose a novel framework to specifically address the larger domain gaps.
arXiv Detail & Related papers (2020-03-08T14:20:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.