DPOD: Domain-Specific Prompt Tuning for Multimodal Fake News Detection

• URL: http://arxiv.org/abs/2311.16496v3
• Date: Wed, 13 Mar 2024 02:32:32 GMT
• Title: DPOD: Domain-Specific Prompt Tuning for Multimodal Fake News Detection
• Authors: Debarshi Brahma, Amartya Bhattacharya, Suraj Nagaje Mahadev, Anmol Asati, Vikas Verma, Soma Biswas
• Abstract summary: Fake news using out-of-context images has become widespread and is a relevant problem in this era of information overload. We explore whether out-of-domain data can help to improve out-of-context misinformation detection of a desired domain. We propose a novel framework termed DPOD (Domain-specific Prompt-tuning using Out-of-Domain data)
• Score: 15.599951180606947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The spread of fake news using out-of-context images has become widespread and is a relevant problem in this era of information overload. Such out-of-context fake news may arise across different domains like politics, sports, entertainment, etc. In practical scenarios, an inherent problem of imbalance exists among news articles from such widely varying domains, resulting in a few domains with abundant data, while the rest containing very limited data. Under such circumstances, it is imperative to develop methods which can work in such varying amounts of data setting. In this work, we explore whether out-of-domain data can help to improve out-of-context misinformation detection (termed here as multi-modal fake news detection) of a desired domain, to address this challenging problem. Towards this goal, we propose a novel framework termed DPOD (Domain-specific Prompt-tuning using Out-of-Domain data). First, to compute generalizable features, we modify the Vision-Language Model, CLIP to extract features that helps to align the representations of the images and corresponding text captions of both the in-domain and out-of-domain data in a label-aware manner. Further, we propose a domain-specific prompt learning technique which leverages the training samples of all the available domains based on the extent they can be useful to the desired domain. Extensive experiments on a large-scale benchmark dataset, namely NewsCLIPpings demonstrate that the proposed framework achieves state of-the-art performance, significantly surpassing the existing approaches for this challenging task. Code will be released on acceptance.

Related papers

• Learning Domain-Invariant Features for Out-of-Context News Detection [19.335065976085982]
  Out-of-context news is a common type of misinformation on online media platforms. In this work, we focus on domain adaptive out-of-context news detection. We propose ConDA-TTA which applies contrastive learning and maximum mean discrepancy (MMD) to learn domain-invariant features.
  arXiv  Detail & Related papers  (2024-06-11T16:34:02Z)
• Beyond Finite Data: Towards Data-free Out-of-distribution Generalization via Extrapolation [19.944946262284123]
  Humans can easily extrapolate novel domains, thus, an intriguing question arises: How can neural networks extrapolate like humans and achieve OOD generalization? We introduce a novel approach to domain extrapolation that leverages reasoning ability and the extensive knowledge encapsulated within large language models (LLMs) to synthesize entirely new domains. Our methods exhibit commendable performance in this setting, even surpassing the supervised setting by approximately 1-2% on datasets such as VLCS.
  arXiv  Detail & Related papers  (2024-03-08T18:44:23Z)
• Robust Domain Misinformation Detection via Multi-modal Feature Alignment [49.89164555394584]
  We propose a robust domain and cross-modal approach for multi-modal misinformation detection. It reduces the domain shift by aligning the joint distribution of textual and visual modalities. We also propose a framework that simultaneously considers application scenarios of domain generalization.
  arXiv  Detail & Related papers  (2023-11-24T07:06:16Z)
• Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration [64.58185031596169]
  Explore-Instruct is a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts.
  arXiv  Detail & Related papers  (2023-10-13T15:03:15Z)
• Harnessing the Power of Multi-Lingual Datasets for Pre-training: Towards Enhancing Text Spotting Performance [15.513912470752041]
  The adaptation capability to a wide range of domains is crucial for scene text spotting models when deployed to real-world conditions. Here, we investigate the problem of domain-adaptive scene text spotting, i.e., training a model on multi-domain source data. The results clearly demonstrate the potential of intermediate representations to achieve significant performance on text spotting benchmarks across multiple domains.
  arXiv  Detail & Related papers  (2023-10-02T06:08:01Z)
• Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval [55.122020263319634]
  Video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query. In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain but the domain of interest only contains unannotated datasets. We propose a novel Multi-Modal Cross-Domain Alignment network to transfer the annotation knowledge from the source domain to the target domain.
  arXiv  Detail & Related papers  (2022-09-23T12:58:20Z)
• Improving Fake News Detection of Influential Domain via Domain- and Instance-Level Transfer [16.886024206337257]
  We propose a Domain- and Instance-level Transfer Framework for Fake News Detection (DITFEND) DITFEND could improve the performance of specific target domains. Online experiments show that it brings additional improvements over the base models in a real-world scenario.
  arXiv  Detail & Related papers  (2022-09-19T10:21:13Z)
• Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting [75.80116276369694]
  In crowd counting, due to the problem of laborious labelling, it is perceived intractability of collecting a new large-scale dataset. We resort to the multi-domain joint learning and propose a simple but effective Domain-specific Knowledge Propagating Network (DKPNet) It is mainly achieved by proposing the novel Variational Attention(VA) technique for explicitly modeling the attention distributions for different domains.
  arXiv  Detail & Related papers  (2021-08-18T08:06:37Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.