Automated Evidence Collection for Fake News Detection

• URL: http://arxiv.org/abs/2112.06507v1
• Date: Mon, 13 Dec 2021 09:38:41 GMT
• Title: Automated Evidence Collection for Fake News Detection
• Authors: Mrinal Rawat, Diptesh Kanojia
• Abstract summary: We propose a novel approach that improves over the current automatic fake news detection approaches. Our approach extracts supporting evidence from the web articles and then selects appropriate text to be treated as evidence sets. Our experiments, using both machine learning and deep learning-based methods, help perform an extensive evaluation of our approach.
• Score: 11.324403127916877
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fake news, misinformation, and unverifiable facts on social media platforms propagate disharmony and affect society, especially when dealing with an epidemic like COVID-19. The task of Fake News Detection aims to tackle the effects of such misinformation by classifying news items as fake or real. In this paper, we propose a novel approach that improves over the current automatic fake news detection approaches by automatically gathering evidence for each claim. Our approach extracts supporting evidence from the web articles and then selects appropriate text to be treated as evidence sets. We use a pre-trained summarizer on these evidence sets and then use the extracted summary as supporting evidence to aid the classification task. Our experiments, using both machine learning and deep learning-based methods, help perform an extensive evaluation of our approach. The results show that our approach outperforms the state-of-the-art methods in fake news detection to achieve an F1-score of 99.25 over the dataset provided for the CONSTRAINT-2021 Shared Task. We also release the augmented dataset, our code and models for any further research.

Related papers

• Search, Examine and Early-Termination: Fake News Detection with Annotation-Free Evidences [32.11238363508177]
  We propose an approach named textbfSEE that retrieves useful information from web-searched annotation-free evidences. The experimental results demonstrate that the proposed method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-07-10T07:22:30Z)
• Adapting Fake News Detection to the Era of Large Language Models [48.5847914481222]
  We study the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news. Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa.
  arXiv  Detail & Related papers  (2023-11-02T08:39:45Z)
• ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media [74.93847489218008]
  We present a novel task, identifying manipulation of news on social media, which aims to detect manipulation in social media posts and identify manipulated or inserted information. To study this task, we have proposed a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles. Our analysis demonstrates that this task is highly challenging, with large language models (LLMs) yielding unsatisfactory performance.
  arXiv  Detail & Related papers  (2023-05-23T16:40:07Z)
• Verifying the Robustness of Automatic Credibility Assessment [50.55687778699995]
  We show that meaning-preserving changes in input text can mislead the models. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions.
  arXiv  Detail & Related papers  (2023-03-14T16:11:47Z)
• Multiverse: Multilingual Evidence for Fake News Detection [71.51905606492376]
  Multiverse is a new feature based on multilingual evidence that can be used for fake news detection. The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed.
  arXiv  Detail & Related papers  (2022-11-25T18:24:17Z)
• Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation [105.20743048379387]
  We propose a novel framework for generating training examples informed by the known styles and strategies of human-authored propaganda. Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles. Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62 - 7.69% F1 score on two public datasets.
  arXiv  Detail & Related papers  (2022-03-10T14:24:19Z)
• A Study of Fake News Reading and Annotating in Social Media Context [1.0499611180329804]
  We present an eye-tracking study, in which we let 44 lay participants to casually read through a social media feed containing posts with news articles, some of which were fake. In a second run, we asked the participants to decide on the truthfulness of these articles. We also describe a follow-up qualitative study with a similar scenario but this time with 7 expert fake news annotators.
  arXiv  Detail & Related papers  (2021-09-26T08:11:17Z)
• A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles [5.979726271522835]
  We describe a novel Fake News Detection system that automatically identifies whether a news item is "real" or "fake" We have used an ensemble model consisting of pre-trained models followed by a statistical feature fusion network. Our proposed framework have also quantified reliable predictive uncertainty along with proper class output confidence level for the classification task.
  arXiv  Detail & Related papers  (2021-04-05T06:35:30Z)
• Supporting verification of news articles with automated search for semantically similar articles [0.0]
  We propose an evidence retrieval approach to handle fake news. The learning task is formulated as an unsupervised machine learning problem. We find that our approach is agnostic to concept drifts, i.e. the machine learning task is independent of the hypotheses in a text.
  arXiv  Detail & Related papers  (2021-03-29T12:56:59Z)
• Connecting the Dots Between Fact Verification and Fake News Detection [21.564628184287173]
  We propose a simple yet effective approach to connect the dots between fact verification and fake news detection. Our approach makes use of the recent success of fact verification models and enables zero-shot fake news detection.
  arXiv  Detail & Related papers  (2020-10-11T09:28:52Z)
• Machine Learning Explanations to Prevent Overtrust in Fake News Detection [64.46876057393703]
  This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news. We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms. For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
  arXiv  Detail & Related papers  (2020-07-24T05:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.