Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens

• URL: http://arxiv.org/abs/2608.21389v1
• Date: Sun, 26 Jul 2026 15:40:41 GMT
• Title: Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens
• Abstract summary: We present empirical findings from a human-subject study in which users classified news fragments by origin (human vs. machine) and veracity (real vs. fake)<n>We organize results using an adapted cybersecurity kill chain as a taxonomy for intervention, mapping perception data onto stages of a cognitive attack lifecycle.<n>Three key findings emerge: (1) a perception-accuracy gap where heightened suspicion does not improve detection; (2) modern LLMs frequently produce human-indistinguishable text; and (3) an asymmetric cognitive fatigue effect where fake-news detection degrades by 10.2 percentage points under sustained exposure while AI-origin detection remains stable.
• Score: 47.03825808787752
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI enables customized misinformation at scale, yet defenses remain largely reactive. We present empirical findings from a human-subject study (n=504 participants, n=2,438 judgments) in which users classified news fragments by origin (human vs. machine) and veracity (real vs. fake). We organize results using an adapted cybersecurity kill chain as a taxonomy for intervention, mapping perception data onto stages of a cognitive attack lifecycle. Three key findings emerge: (1) a perception-accuracy gap where heightened suspicion does not improve detection; (2) modern LLMs frequently produce human-indistinguishable text; and (3) an asymmetric cognitive fatigue effect where fake-news detection degrades by 10.2 percentage points under sustained exposure while AI-origin detection remains stable. These findings identify candidate intervention points for proactive defense against AI-driven disinformation.

Related papers

• Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers [0.0]
  We introduce an automated evaluation framework adapted from the Honeyquest instrument to assess LLM attacker judgment at scale.<n>Our 21-LLM cohort spanned 10 providers, diverse architectures and specializations, open- and closed-weight models, and parameter scales from 8B to over 1T.<n>Ultimately, these findings demonstrate that human-centered deception hypotheses do not reliably transfer to AI attackers.
  arXiv  Detail & Related papers  (2026-06-19T02:01:50Z)
• Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News [47.03825808787752]
  Can humans tell whether a news article was written by a person or a large language model (LLM)?<n>We investigate this question using JudgeGPT, a study platform that measures source attribution (human vs. machine) and authenticity judgment (legitimate vs. fake) on continuous scales.<n>We report five findings: (1) participants cannot reliably distinguish machine-generated from human-written text; (2) this inability holds across all tested models, including open-weight models with as few as 7B parameters; and (3) accuracy degrades after approximately 30 sequential evaluations due to cognitive fatigue.
  arXiv  Detail & Related papers  (2026-04-04T15:03:42Z)
• Can You Tell It's AI? Human Perception of Synthetic Voices in Vishing Scenarios [3.2976205772213123]
  Large Language Models and commercial speech synthesis systems now enable highly realistic AI-generated voice scams (vishing)<n>Yet it remains unclear whether individuals can reliably distinguish AI-generated speech from human-recorded voices in realistic scam contexts.<n>We conducted a controlled online study in which 22 participants evaluated 16 vishing-style audio clips and classified each as human or AI.
  arXiv  Detail & Related papers  (2026-02-23T17:17:53Z)
• The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance [0.0]
  Large language model (LLM)-based conversational AI systems present a challenge to human cognition.<n>This paper proposes that a significant epistemic risk from conversational AI may lie not in inaccuracy or intentional deception, but in something more fundamental.
  arXiv  Detail & Related papers  (2026-01-11T22:28:56Z)
• Investigating Human-Identifiable Features Hidden in Adversarial Perturbations [54.39726653562144]
  Our study explores up to five attack algorithms across three datasets. We identify human-identifiable features in adversarial perturbations. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models.
  arXiv  Detail & Related papers  (2023-09-28T22:31:29Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Towards Adversarial Patch Analysis and Certified Defense against Crowd Counting [61.99564267735242]
  Crowd counting has drawn much attention due to its importance in safety-critical surveillance systems. Recent studies have demonstrated that deep neural network (DNN) methods are vulnerable to adversarial attacks. We propose a robust attack strategy called Adversarial Patch Attack with Momentum to evaluate the robustness of crowd counting models.
  arXiv  Detail & Related papers  (2021-04-22T05:10:55Z)
• Adversarial vs behavioural-based defensive AI with joint, continual and active learning: automated evaluation of robustness to deception, poisoning and concept drift [62.997667081978825]
  Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security. In this paper, we present a solution to effectively mitigate this attack by improving the detection process and efficiently leveraging human expertise.
  arXiv  Detail & Related papers  (2020-01-13T13:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.