A Revisit of Fake News Dataset with Augmented Fact-checking by ChatGPT
- URL: http://arxiv.org/abs/2312.11870v1
- Date: Tue, 19 Dec 2023 05:46:11 GMT
- Title: A Revisit of Fake News Dataset with Augmented Fact-checking by ChatGPT
- Authors: Zizhong Li, Haopeng Zhang, Jiawei Zhang
- Abstract summary: Existing fake news detection datasets are sourced from human journalists.
In this paper, we revisit the existing fake news dataset verified by human journalists with augmented fact-checking by large language models.
- Score: 8.363702038073814
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The proliferation of fake news has emerged as a critical issue in recent
years, requiring significant efforts to detect it. However, the existing fake
news detection datasets are sourced from human journalists, which are likely to
have inherent bias limitations due to the highly subjective nature of this
task. In this paper, we revisit the existing fake news dataset verified by
human journalists with augmented fact-checking by large language models
(ChatGPT), and we name the augmented fake news dataset ChatGPT-FC. We
quantitatively analyze the distinctions and resemblances between human
journalists and LLM in assessing news subject credibility, news creator
credibility, time-sensitive, and political framing. Our findings highlight
LLM's potential to serve as a preliminary screening method, offering a
promising avenue to mitigate the inherent biases of human journalists and
enhance fake news detection.
Related papers
- Adapting Fake News Detection to the Era of Large Language Models [48.5847914481222]
We study the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news.
Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa.
arXiv Detail & Related papers (2023-11-02T08:39:45Z) - Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News
Detection [50.07850264495737]
"Prompt-and-Align" (P&A) is a novel prompt-based paradigm for few-shot fake news detection.
We show that P&A sets new states-of-the-art for few-shot fake news detection performance by significant margins.
arXiv Detail & Related papers (2023-09-28T13:19:43Z) - fakenewsbr: A Fake News Detection Platform for Brazilian Portuguese [0.6775616141339018]
This paper presents a comprehensive study on detecting fake news in Brazilian Portuguese.
We propose a machine learning-based approach that leverages natural language processing techniques, including TF-IDF and Word2Vec.
We develop a user-friendly web platform, fakenewsbr.com, to facilitate the verification of news articles' veracity.
arXiv Detail & Related papers (2023-09-20T04:10:03Z) - Fake News Detectors are Biased against Texts Generated by Large Language
Models [39.36284616311687]
The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society.
We present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation.
arXiv Detail & Related papers (2023-09-15T18:04:40Z) - Unveiling the Hidden Agenda: Biases in News Reporting and Consumption [59.55900146668931]
We build a six-year dataset on the Italian vaccine debate and adopt a Bayesian latent space model to identify narrative and selection biases.
We found a nonlinear relationship between biases and engagement, with higher engagement for extreme positions.
Analysis of news consumption on Twitter reveals common audiences among news outlets with similar ideological positions.
arXiv Detail & Related papers (2023-01-14T18:58:42Z) - Nothing Stands Alone: Relational Fake News Detection with Hypergraph
Neural Networks [49.29141811578359]
We propose to leverage a hypergraph to represent group-wise interaction among news, while focusing on important news relations with its dual-level attention mechanism.
Our approach yields remarkable performance and maintains the high performance even with a small subset of labeled news data.
arXiv Detail & Related papers (2022-12-24T00:19:32Z) - Faking Fake News for Real Fake News Detection: Propaganda-loaded
Training Data Generation [105.20743048379387]
We propose a novel framework for generating training examples informed by the known styles and strategies of human-authored propaganda.
Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles.
Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62 - 7.69% F1 score on two public datasets.
arXiv Detail & Related papers (2022-03-10T14:24:19Z) - Stance Detection with BERT Embeddings for Credibility Analysis of
Information on Social Media [1.7616042687330642]
We propose a model for detecting fake news using stance as one of the features along with the content of the article.
Our work interprets the content with automatic feature extraction and the relevance of the text pieces.
The experiment conducted on the real-world dataset indicates that our model outperforms the previous work and enables fake news detection with an accuracy of 95.32%.
arXiv Detail & Related papers (2021-05-21T10:46:43Z) - User Preference-aware Fake News Detection [61.86175081368782]
Existing fake news detection algorithms focus on mining news content for deceptive signals.
We propose a new framework, UPFD, which simultaneously captures various signals from user preferences by joint content and graph modeling.
arXiv Detail & Related papers (2021-04-25T21:19:24Z) - Early Detection of Fake News by Utilizing the Credibility of News,
Publishers, and Users Based on Weakly Supervised Learning [23.96230360460216]
We propose a novel Structure-aware Multi-head Attention Network (SMAN), which combines the news content, publishing, and reposting relations of publishers and users.
SMAN can detect fake news in 4 hours with an accuracy of over 91%, which is much faster than the state-of-the-art models.
arXiv Detail & Related papers (2020-12-08T05:53:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.