Do people rely on ChatGPT more than their peers to detect deepfake news?

• URL: http://arxiv.org/abs/2608.01540v1
• Date: Sun, 02 Aug 2026 23:24:44 GMT
• Title: Do people rely on ChatGPT more than their peers to detect deepfake news?
• Abstract summary: This study investigates how people rely on different sources of advice when detecting AI-generated fake news (deepfake news)<n>The results show that participants rely more on ChatGPT than on human peers when detecting GPT-2-generated deepfake news.<n>In an additional experiment conducted in 2025 under the same experimental procedure, participants relied more on linguistic experts than on ChatGPT.
• Score: 0.1912196187222047
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This experimental study investigates how people rely on different sources of advice when detecting AI-generated fake news (deepfake news). In a laboratory deepfake detection task, student participants identified the proportion of human-written (non-AI-generated) content in synthetic deepfake news articles and received advice from ChatGPT (GPT-4), human peers, or linguistic experts. The results show that participants rely more on ChatGPT than on human peers when detecting GPT-2-generated deepfake news. Participants also rely more on linguistic experts than on peers, while the relative reliance on experts versus ChatGPT is mixed across experimental waves, potentially reflecting time trends in beliefs about AI-based detection. Importantly, in the additional experiment conducted in 2025 under the same experimental procedure, participants relied more on linguistic experts than on ChatGPT. Moreover, performance improvements reflect the joint role of reliance and advice quality, arising primarily when participants rely on high-quality advice. Overall, relying on AI to detect AI-generated deepfakes can improve detection outcomes, but only when AI-based detection tools are of sufficiently high quality. These findings highlight the dual role of GAI as both a source of deepfakes and a tool for mitigating related risks.

Related papers

• Deepfake tweets automatic detection [0.0]
  This study uses advanced natural language processing (NLP) techniques to distinguish between genuine and AI-generated texts. By developing reliable methods for detecting AI-generated misinformation, this work contributes to a more trustworthy online information environment.
  arXiv  Detail & Related papers  (2024-06-24T09:55:31Z)
• DEMASQ: Unmasking the ChatGPT Wordsmith [63.8746084667206]
  We propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. Our method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machine-generated content and (ii) the alterations made by humans to evade previous detection methods.
  arXiv  Detail & Related papers  (2023-11-08T21:13:05Z)
• Do You Trust ChatGPT? -- Perceived Credibility of Human and AI-Generated Content [0.8602553195689513]
  This paper examines how individuals perceive the credibility of content originating from human authors versus content generated by large language models. Surprisingly, our results demonstrate that regardless of the user interface presentation, participants tend to attribute similar levels of credibility. Participants also do not report any different perceptions of competence and trustworthiness between human and AI-generated content.
  arXiv  Detail & Related papers  (2023-09-05T18:29:29Z)
• Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text [48.36706154871577]
  We introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts) It diverges from extant corpora by comprising pairs of human-written and ChatGPT-polished abstracts instead of purely ChatGPT-generated texts. We also propose the "Polish Ratio" method, an innovative measure of the degree of modification made by ChatGPT compared to the original human-written text.
  arXiv  Detail & Related papers  (2023-07-21T06:38:37Z)
• On the Possibilities of AI-Generated Text Detection [76.55825911221434]
  We argue that as machine-generated text approximates human-like quality, the sample size needed for detection bounds increases. We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero.
  arXiv  Detail & Related papers  (2023-04-10T17:47:39Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts? [27.700129124128747]
  Collaboration among humans can potentially improve the detection of deepfake texts. The strongest indicator of deepfake texts is their lack of coherence and consistency.
  arXiv  Detail & Related papers  (2023-04-03T14:06:47Z)
• Consistency Analysis of ChatGPT [65.268245109828]
  This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions.
  arXiv  Detail & Related papers  (2023-03-11T01:19:01Z)
• On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective [67.98821225810204]
  We evaluate the robustness of ChatGPT from the adversarial and out-of-distribution perspective. Results show consistent advantages on most adversarial and OOD classification and translation tasks. ChatGPT shows astounding performance in understanding dialogue-related texts.
  arXiv  Detail & Related papers  (2023-02-22T11:01:20Z)
• Human Perception of Audio Deepfakes [6.40753664615445]
  We present results from comparing the abilities of humans and machines for detecting audio deepfakes. In our experiment, 472 unique users competed against a state-of-the-art AI deepfake detection algorithm for 14912 total rounds of the game. We find that humans and deepfake detection algorithms share similar strengths and weaknesses, both struggling to detect certain types of attacks.
  arXiv  Detail & Related papers  (2021-07-20T09:19:42Z)
• Machine Learning Explanations to Prevent Overtrust in Fake News Detection [64.46876057393703]
  This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news. We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms. For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
  arXiv  Detail & Related papers  (2020-07-24T05:42:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.