Findings of Factify 2: Multimodal Fake News Detection

• URL: http://arxiv.org/abs/2307.10475v2
• Date: Tue, 12 Sep 2023 18:51:05 GMT
• Title: Findings of Factify 2: Multimodal Fake News Detection
• Authors: S Suryavardan, Shreyash Mishra, Megha Chakraborty, Parth Patwa, Anku Rani, Aman Chadha, Aishwarya Reganti, Amitava Das, Amit Sheth, Manoj Chinnakotla, Asif Ekbal, Srijan Kumar
• Abstract summary: We present the outcome of the Factify 2 shared task, which provides a multi-modal fact verification and satire news dataset. The data calls for a comparison based approach to the task by pairing social media claims with supporting documents, with both text and image, divided into 5 classes based on multi-modal relations. The highest F1 score averaged for all five classes was 81.82%.
• Score: 36.34201719103715
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With social media usage growing exponentially in the past few years, fake news has also become extremely prevalent. The detrimental impact of fake news emphasizes the need for research focused on automating the detection of false information and verifying its accuracy. In this work, we present the outcome of the Factify 2 shared task, which provides a multi-modal fact verification and satire news dataset, as part of the DeFactify 2 workshop at AAAI'23. The data calls for a comparison based approach to the task by pairing social media claims with supporting documents, with both text and image, divided into 5 classes based on multi-modal relations. In the second iteration of this task we had over 60 participants and 9 final test-set submissions. The best performances came from the use of DeBERTa for text and Swinv2 and CLIP for image. The highest F1 score averaged for all five classes was 81.82%.

Related papers

• Overview of Factify5WQA: Fact Verification through 5W Question-Answering [25.679348027485254]
  The Factify5WQA task aims to increase research towards automated fake news detection. The best performing team posted an accuracy of 69.56%, which is a near 35% improvement over the baseline.
  arXiv  Detail & Related papers  (2024-10-05T17:28:18Z)
• ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media [74.93847489218008]
  We present a novel task, identifying manipulation of news on social media, which aims to detect manipulation in social media posts and identify manipulated or inserted information. To study this task, we have proposed a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles. Our analysis demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance.
  arXiv  Detail & Related papers  (2023-05-23T16:40:07Z)
• Factify 2: A Multimodal Fake News and Satire News Dataset [36.34201719103715]
  We provide a multi-modal fact-checking dataset called FACTIFY 2, improving Factify 1 by using new data sources and adding satire articles. Similar to FACTIFY 1.0, we have three broad categories - support, no-evidence, and refute, with sub-categories based on the entailment of visual and textual data. We also provide a BERT and Vison Transformer based baseline, which achieves 65% F1 score in the test set.
  arXiv  Detail & Related papers  (2023-04-08T03:14:19Z)
• INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification [3.5408382050384883]
  We propose a structure coherence-based multi-modal fact verification scheme to classify fake news. Our weighted average F1 score has reached 0.8079, achieving 2nd place in FACTIFY2.
  arXiv  Detail & Related papers  (2023-03-02T11:18:56Z)
• Multiverse: Multilingual Evidence for Fake News Detection [71.51905606492376]
  Multiverse is a new feature based on multilingual evidence that can be used for fake news detection. The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed.
  arXiv  Detail & Related papers  (2022-11-25T18:24:17Z)
• UrduFake@FIRE2020: Shared Track on Fake News Identification in Urdu [62.6928395368204]
  This paper gives the overview of the first shared task at FIRE 2020 on fake news detection in the Urdu language. The goal is to identify fake news using a dataset composed of 900 annotated news articles for training and 400 news articles for testing. The dataset contains news in five domains: (i) Health, (ii) Sports, (iii) Showbiz, (iv) Technology, and (v) Business.
  arXiv  Detail & Related papers  (2022-07-25T03:46:51Z)
• Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2020 [62.6928395368204]
  Task was posed as a binary classification task, in which the goal is to differentiate between real and fake news. We provided a dataset divided into 900 annotated news articles for training and 400 news articles for testing. 42 teams from 6 different countries (India, China, Egypt, Germany, Pakistan, and the UK) registered for the task.
  arXiv  Detail & Related papers  (2022-07-25T03:41:32Z)
• Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2021 [55.41644538483948]
  The goal of the shared task is to motivate the community to come up with efficient methods for solving this vital problem. The training set contains 1300 annotated news articles -- 750 real news, 550 fake news, while the testing set contains 300 news articles -- 200 real, 100 fake news. The best performing system obtained an F1-macro score of 0.679, which is lower than the past year's best result of 0.907 F1-macro.
  arXiv  Detail & Related papers  (2022-07-11T18:58:36Z)
• UofA-Truth at Factify 2022 : Transformer And Transfer Learning Based Multi-Modal Fact-Checking [0.0]
  We attempted to tackle the problem of automated misinformation/disinformation detection in multi-modal news sources. Our model produced an F1-weighted score of 74.807%, which was the fourth best out of all the submissions.
  arXiv  Detail & Related papers  (2022-01-28T18:13:03Z)
• Multimodal Fake News Detection [1.929039244357139]
  We perform a fine-grained classification of fake news on the Fakeddit dataset using both unimodal and multimodal approaches. Some fake news categories such as Manipulated content, Satire or False connection strongly benefit from the use of images. Using images also improves the results of the other categories, but with less impact.
  arXiv  Detail & Related papers  (2021-12-09T10:57:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.