FakeGPT: Fake News Generation, Explanation and Detection of Large Language Models

• URL: http://arxiv.org/abs/2310.05046v2
• Date: Sat, 6 Apr 2024 08:10:43 GMT
• Title: FakeGPT: Fake News Generation, Explanation and Detection of Large Language Models
• Authors: Yue Huang, Lichao Sun,
• Abstract summary: ChatGPT has gained significant attention due to its exceptional natural language processing capabilities. We employ four prompt methods to generate fake news samples and prove the high quality of these samples through both self-assessment and human evaluation. We examine ChatGPT's capacity to identify fake news and propose a reason-aware prompt method to improve its performance.
• Score: 18.543917359268345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rampant spread of fake news has adversely affected society, resulting in extensive research on curbing its spread. As a notable milestone in large language models (LLMs), ChatGPT has gained significant attention due to its exceptional natural language processing capabilities. In this study, we present a thorough exploration of ChatGPT's proficiency in generating, explaining, and detecting fake news as follows. Generation -- We employ four prompt methods to generate fake news samples and prove the high quality of these samples through both self-assessment and human evaluation. Explanation -- We obtain nine features to characterize fake news based on ChatGPT's explanations and analyze the distribution of these factors across multiple public datasets. Detection -- We examine ChatGPT's capacity to identify fake news. We explore its detection consistency and then propose a reason-aware prompt method to improve its performance. Although our experiments demonstrate that ChatGPT shows commendable performance in detecting fake news, there is still room for its improvement. Consequently, we further probe into the potential extra information that could bolster its effectiveness in detecting fake news.

Related papers

• Detection of Human and Machine-Authored Fake News in Urdu [2.013675429941823]
  Social media has amplified the spread of fake news. Traditional fake news detection methods relying on linguistic cues become less effective. We propose a hierarchical detection strategy to improve the accuracy and robustness.
  arXiv  Detail & Related papers  (2024-10-25T12:42:07Z)
• Exploring the Deceptive Power of LLM-Generated Fake News: A Study of Real-World Detection Challenges [21.425647152424585]
  We propose a strong fake news attack method called conditional Variational-autoencoder-Like Prompt (VLPrompt) Unlike current methods, VLPrompt eliminates the need for additional data collection while maintaining contextual coherence. Our experiments, including various detection methods and novel human study metrics, were conducted to assess their performance on our dataset.
  arXiv  Detail & Related papers  (2024-03-27T04:39:18Z)
• Adapting Fake News Detection to the Era of Large Language Models [48.5847914481222]
  We study the interplay between machine-(paraphrased) real news, machine-generated fake news, human-written fake news, and human-written real news. Our experiments reveal an interesting pattern that detectors trained exclusively on human-written articles can indeed perform well at detecting machine-generated fake news, but not vice versa.
  arXiv  Detail & Related papers  (2023-11-02T08:39:45Z)
• fakenewsbr: A Fake News Detection Platform for Brazilian Portuguese [0.6775616141339018]
  This paper presents a comprehensive study on detecting fake news in Brazilian Portuguese. We propose a machine learning-based approach that leverages natural language processing techniques, including TF-IDF and Word2Vec. We develop a user-friendly web platform, fakenewsbr.com, to facilitate the verification of news articles' veracity.
  arXiv  Detail & Related papers  (2023-09-20T04:10:03Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Multiverse: Multilingual Evidence for Fake News Detection [71.51905606492376]
  Multiverse is a new feature based on multilingual evidence that can be used for fake news detection. The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed.
  arXiv  Detail & Related papers  (2022-11-25T18:24:17Z)
• A Multi-Policy Framework for Deep Learning-Based Fake News Detection [0.31498833540989407]
  This work introduces Multi-Policy Statement Checker (MPSC), a framework that automates fake news detection. MPSC uses deep learning techniques to analyze a statement itself and its related news articles, predicting whether it is seemingly credible or suspicious.
  arXiv  Detail & Related papers  (2022-06-01T21:25:21Z)
• Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation [105.20743048379387]
  We propose a novel framework for generating training examples informed by the known styles and strategies of human-authored propaganda. Specifically, we perform self-critical sequence training guided by natural language inference to ensure the validity of the generated articles. Our experimental results show that fake news detectors trained on PropaNews are better at detecting human-written disinformation by 3.62 - 7.69% F1 score on two public datasets.
  arXiv  Detail & Related papers  (2022-03-10T14:24:19Z)
• User Preference-aware Fake News Detection [61.86175081368782]
  Existing fake news detection algorithms focus on mining news content for deceptive signals. We propose a new framework, UPFD, which simultaneously captures various signals from user preferences by joint content and graph modeling.
  arXiv  Detail & Related papers  (2021-04-25T21:19:24Z)
• Machine Learning Explanations to Prevent Overtrust in Fake News Detection [64.46876057393703]
  This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news. We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms. For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
  arXiv  Detail & Related papers  (2020-07-24T05:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.