Detecting COVID-19 Conspiracy Theories with Transformers and TF-IDF

• URL: http://arxiv.org/abs/2205.00377v1
• Date: Sun, 1 May 2022 01:48:48 GMT
• Title: Detecting COVID-19 Conspiracy Theories with Transformers and TF-IDF
• Authors: Haoming Guo, Tianyi Huang, Huixuan Huang, Mingyue Fan, Gerald Friedland
• Abstract summary: We present our methods and results for three fake news detection tasks at MediaEval benchmark 2021. We find that a pre-trained transformer yields the best validation results, but a randomly trained transformer with smart design can also be trained to reach accuracies close to that of the pre-trained transformer.
• Score: 2.3202611780303553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The sharing of fake news and conspiracy theories on social media has wide-spread negative effects. By designing and applying different machine learning models, researchers have made progress in detecting fake news from text. However, existing research places a heavy emphasis on general, common-sense fake news, while in reality fake news often involves rapidly changing topics and domain-specific vocabulary. In this paper, we present our methods and results for three fake news detection tasks at MediaEval benchmark 2021 that specifically involve COVID-19 related topics. We experiment with a group of text-based models including Support Vector Machines, Random Forest, BERT, and RoBERTa. We find that a pre-trained transformer yields the best validation results, but a randomly initialized transformer with smart design can also be trained to reach accuracies close to that of the pre-trained transformer.

Related papers

• Adapting Fake News Detection to the Era of Large Language Models [48.5847914481222]
  We study the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news. Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa.
  arXiv  Detail & Related papers  (2023-11-02T08:39:45Z)
• Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News Detection [50.07850264495737]
  "Prompt-and-Align" (P&A) is a novel prompt-based paradigm for few-shot fake news detection. We show that P&A sets new states-of-the-art for few-shot fake news detection performance by significant margins.
  arXiv  Detail & Related papers  (2023-09-28T13:19:43Z)
• Performance Analysis of Transformer Based Models (BERT, ALBERT and RoBERTa) in Fake News Detection [0.0]
  Top three areas most exposed to hoaxes and misinformation by residents are in Banten, DKI Jakarta and West Java. Previous study indicates a superior performance of a transformer model known as BERT over and above non transformer approach. In this research, we explore those transformer models and found that ALBERT outperformed other models with 87.6% accuracy, 86.9% precision, 86.9% F1-score, and 174.5 run-time (s/epoch) respectively.
  arXiv  Detail & Related papers  (2023-08-09T13:33:27Z)
• MisRoB{\AE}RTa: Transformers versus Misinformation [0.6091702876917281]
  We propose a novel transformer-based deep neural ensemble architecture for misinformation detection. MisRoBAERTa takes advantage of two transformers (BART & RoBERTa) to improve the classification performance. For training and testing, we used a large real-world news articles dataset labeled with 10 classes.
  arXiv  Detail & Related papers  (2023-04-16T12:14:38Z)
• Verifying the Robustness of Automatic Credibility Assessment [50.55687778699995]
  We show that meaning-preserving changes in input text can mislead the models. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions.
  arXiv  Detail & Related papers  (2023-03-14T16:11:47Z)
• Multiverse: Multilingual Evidence for Fake News Detection [71.51905606492376]
  Multiverse is a new feature based on multilingual evidence that can be used for fake news detection. The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed.
  arXiv  Detail & Related papers  (2022-11-25T18:24:17Z)
• Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation [105.20743048379387]
  We propose a novel framework for generating training examples informed by the known styles and strategies of human-authored propaganda. Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles. Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62 - 7.69% F1 score on two public datasets.
  arXiv  Detail & Related papers  (2022-03-10T14:24:19Z)
• Transforming Fake News: Robust Generalisable News Classification Using Transformers [8.147652597876862]
  Using the publicly available ISOT and Combined Corpus datasets, this study explores transformers' abilities to identify fake news. We propose a novel two-step classification pipeline to remove such articles from both model training and the final deployed inference system. Experiments over the ISOT and Combined Corpus datasets show that transformers achieve an increase in F1 scores of up to 4.9% for out of distribution generalisation.
  arXiv  Detail & Related papers  (2021-09-20T19:03:16Z)
• Transformer based Automatic COVID-19 Fake News Detection System [9.23545668304066]
  Misinformation is especially prevalent in the ongoing coronavirus disease (COVID-19) pandemic. We report a methodology to analyze the reliability of information shared on social media pertaining to the COVID-19 pandemic. Our system obtained 0.9855 f1-score on testset and ranked 5th among 160 teams.
  arXiv  Detail & Related papers  (2021-01-01T06:49:27Z)
• Two Stage Transformer Model for COVID-19 Fake News Detection and Fact Checking [0.3441021278275805]
  We develop a two stage automated pipeline for COVID-19 fake news detection using state of the art machine learning models for natural language processing. The first model leverages a novel fact checking algorithm that retrieves the most relevant facts concerning user claims about particular COVID-19 claims. The second model verifies the level of truth in the claim by computing the textual entailment between the claim and the true facts retrieved from a manually curated COVID-19 dataset.
  arXiv  Detail & Related papers  (2020-11-26T11:50:45Z)
• Machine Learning Explanations to Prevent Overtrust in Fake News Detection [64.46876057393703]
  This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news. We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms. For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
  arXiv  Detail & Related papers  (2020-07-24T05:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.