Performance Analysis of Transformer Based Models (BERT, ALBERT and
RoBERTa) in Fake News Detection
- URL: http://arxiv.org/abs/2308.04950v1
- Date: Wed, 9 Aug 2023 13:33:27 GMT
- Title: Performance Analysis of Transformer Based Models (BERT, ALBERT and
RoBERTa) in Fake News Detection
- Authors: Shafna Fitria Nur Azizah, Hasan Dwi Cahyono, Sari Widya Sihwi, Wisnu
Widiarto
- Abstract summary: Top three areas most exposed to hoaxes and misinformation by residents are in Banten, DKI Jakarta and West Java.
Previous study indicates a superior performance of a transformer model known as BERT over and above non transformer approach.
In this research, we explore those transformer models and found that ALBERT outperformed other models with 87.6% accuracy, 86.9% precision, 86.9% F1-score, and 174.5 run-time (s/epoch) respectively.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Fake news is fake material in a news media format but is not processed
properly by news agencies. The fake material can provoke or defame significant
entities or individuals or potentially even for the personal interests of the
creators, causing problems for society. Distinguishing fake news and real news
is challenging due to limited of domain knowledge and time constraints.
According to the survey, the top three areas most exposed to hoaxes and
misinformation by residents are in Banten, DKI Jakarta and West Java. The model
of transformers is referring to an approach in the field of artificial
intelligence (AI) in natural language processing utilizing the deep learning
architectures. Transformers exercise a powerful attention mechanism to process
text in parallel and produce rich and contextual word representations. A
previous study indicates a superior performance of a transformer model known as
BERT over and above non transformer approach. However, some studies suggest the
performance can be improved with the use of improved BERT models known as
ALBERT and RoBERTa. However, the modified BERT models are not well explored for
detecting fake news in Bahasa Indonesia. In this research, we explore those
transformer models and found that ALBERT outperformed other models with 87.6%
accuracy, 86.9% precision, 86.9% F1-score, and 174.5 run-time (s/epoch)
respectively. Source code available at:
https://github.com/Shafna81/fakenewsdetection.git
Related papers
- Adapting Fake News Detection to the Era of Large Language Models [48.5847914481222]
We study the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news.
Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa.
arXiv Detail & Related papers (2023-11-02T08:39:45Z) - Transformer-based approaches to Sentiment Detection [55.41644538483948]
We examined the performance of four different types of state-of-the-art transformer models for text classification.
The RoBERTa transformer model performs best on the test dataset with a score of 82.6% and is highly recommended for quality predictions.
arXiv Detail & Related papers (2023-03-13T17:12:03Z) - Multiverse: Multilingual Evidence for Fake News Detection [71.51905606492376]
Multiverse is a new feature based on multilingual evidence that can be used for fake news detection.
The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed.
arXiv Detail & Related papers (2022-11-25T18:24:17Z) - Detecting COVID-19 Conspiracy Theories with Transformers and TF-IDF [2.3202611780303553]
We present our methods and results for three fake news detection tasks at MediaEval benchmark 2021.
We find that a pre-trained transformer yields the best validation results, but a randomly trained transformer with smart design can also be trained to reach accuracies close to that of the pre-trained transformer.
arXiv Detail & Related papers (2022-05-01T01:48:48Z) - Faking Fake News for Real Fake News Detection: Propaganda-loaded
Training Data Generation [105.20743048379387]
We propose a novel framework for generating training examples informed by the known styles and strategies of human-authored propaganda.
Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles.
Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62 - 7.69% F1 score on two public datasets.
arXiv Detail & Related papers (2022-03-10T14:24:19Z) - COVID-19 Fake News Detection Using Bidirectional Encoder Representations
from Transformers Based Models [16.400631119118636]
COVID-19 fake news detection has become a novel and important task in the NLP field.
In this paper, we fine tune the pre-trained Bidirectional Representations from Transformers (BERT) model as our base model.
We add BiLSTM layers and CNN layers on the top of the finetuned BERT model with frozen parameters or not frozen parameters methods respectively.
arXiv Detail & Related papers (2021-09-30T02:50:05Z) - Two Stage Transformer Model for COVID-19 Fake News Detection and Fact
Checking [0.3441021278275805]
We develop a two stage automated pipeline for COVID-19 fake news detection using state of the art machine learning models for natural language processing.
The first model leverages a novel fact checking algorithm that retrieves the most relevant facts concerning user claims about particular COVID-19 claims.
The second model verifies the level of truth in the claim by computing the textual entailment between the claim and the true facts retrieved from a manually curated COVID-19 dataset.
arXiv Detail & Related papers (2020-11-26T11:50:45Z) - Connecting the Dots Between Fact Verification and Fake News Detection [21.564628184287173]
We propose a simple yet effective approach to connect the dots between fact verification and fake news detection.
Our approach makes use of the recent success of fact verification models and enables zero-shot fake news detection.
arXiv Detail & Related papers (2020-10-11T09:28:52Z) - Viable Threat on News Reading: Generating Biased News Using Natural
Language Models [49.90665530780664]
We show that publicly available language models can reliably generate biased news content based on an input original news.
We also show that a large number of high-quality biased news articles can be generated using controllable text generation.
arXiv Detail & Related papers (2020-10-05T16:55:39Z) - Machine Learning Explanations to Prevent Overtrust in Fake News
Detection [64.46876057393703]
This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news.
We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms.
For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
arXiv Detail & Related papers (2020-07-24T05:42:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.