Scheming in the wild: detecting real-world AI scheming incidents with open-source intelligence

• URL: http://arxiv.org/abs/2604.09104v1
• Date: Fri, 10 Apr 2026 08:37:18 GMT
• Title: Scheming in the wild: detecting real-world AI scheming incidents with open-source intelligence
• Abstract summary: This paper introduces a novel open-source intelligence (OSINT) methodology for detecting real-world scheming incidents.<n>Analysing over 183,420 transcripts from X (formerly Twitter), we identify 698 real-world scheming-related incidents between October 2025 and March 2026.<n>We find evidence of multiple scheming-related behaviours in real-world deployments previously reported only in experiments.
• Score: 0.09558392439655013
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scheming, the covert pursuit of misaligned goals by AI systems, represents a potentially catastrophic risk, yet scheming research suffers from significant limitations. In particular, scheming evaluations demonstrate behaviours that may not occur in real-world settings, limiting scientific understanding, hindering policy development, and not enabling real-time detection of loss of control incidents. Real-world evidence is needed, but current monitoring techniques are not effective for this purpose. This paper introduces a novel open-source intelligence (OSINT) methodology for detecting real-world scheming incidents: collecting and analysing transcripts from chatbot conversations or command-line interactions shared online. Analysing over 183,420 transcripts from X (formerly Twitter), we identify 698 real-world scheming-related incidents between October 2025 and March 2026. We observe a statistically significant 4.9x increase in monthly incidents from the first to last month, compared to a 1.7x increase in posts discussing scheming. We find evidence of multiple scheming-related behaviours in real-world deployments previously reported only in experiments, many resulting in real-world harms. While we did not detect catastrophic scheming incidents, the behaviours observed demonstrate concerning precursors, such as willingness to disregard instructions, circumvent safeguards, lie to users, and single-mindedly pursue goals in harmful ways. As AI systems become more capable, these could evolve into more strategic scheming with potentially catastrophic consequences. Our findings demonstrate the viability of transcript-based OSINT as a scalable approach to real-world scheming detection supporting scientific research, policy development, and emergency response. We recommend further investment towards OSINT techniques for monitoring scheming and loss of control.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• The Biosecurity Blind Spot: Systematic Dual-use Detection in Open Science Infrastructure [0.6124773188525718]
  We present the first systematic analysis of dual-use research of concern (DURC) content on open preprint servers.<n>Our analysis reveals that dual-use-adjacent knowledge is routinely present in openly accessible titles and abstracts.<n>We argue that institutional review processes, funding requirements, and preprint platform policies must evolve to incorporate proactive, metadata-level monitoring.
  arXiv  Detail & Related papers  (2026-05-10T16:17:41Z)
• From Stealthy Data Fabrication to Unsafe Driving: Realistic Scenario Attacks on Collaborative Perception [7.889284710970368]
  Collaborative perception allows connected and autonomous vehicles to improve perception by sharing sensory data.<n>Prior work shows that attackers can spoof or remove objects by fabricating shared data, yet the practicality of such attacks in real-world driving remains unclear.<n>We present a stealthy, scenario-realistic data fabrication attack that induces unsafe driving behaviors through end-to-end system effects.
  arXiv  Detail & Related papers  (2026-05-02T07:21:22Z)
• Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses [168.50301366360344]
  Embodied Artificial Intelligence (Embodied AI) integrates perception, cognition, planning, and interaction into agents that operate in safety-critical environments.<n>Unlike digital AI systems, embodied agents must act under uncertain sensing, incomplete knowledge, and dynamic human-robot interactions.<n>This survey provides a comprehensive review of safety research in embodied AI, examining attacks and defenses across the full embodied pipeline.
  arXiv  Detail & Related papers  (2026-03-28T13:21:44Z)
• Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems [47.03825808787752]
  This paper transitions from literature review to practical countermeasures.<n>We report on improved AI-generated content through Large Language Models (LLMs) and multimodal systems.<n>We discuss mitigation strategies including LLM-based detection, inoculation approaches, and the dual-use nature of generative AI.
  arXiv  Detail & Related papers  (2026-01-29T16:42:22Z)
• AI Deception: Risks, Dynamics, and Controls [153.71048309527225]
  This project provides a comprehensive and up-to-date overview of the AI deception field.<n>We identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception.<n>We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment.
  arXiv  Detail & Related papers  (2025-11-27T16:56:04Z)
• Anomalous Decision Discovery using Inverse Reinforcement Learning [3.3675535571071746]
  Anomaly detection plays a critical role in Autonomous Vehicles (AVs) by identifying unusual behaviors through perception systems.<n>Current approaches, which often rely on predefined thresholds or supervised learning paradigms, exhibit reduced efficacy when confronted with unseen scenarios.<n>We present Trajectory-Reward Guided Adaptive Pre-training (TRAP), a novel IRL framework for anomaly detection.
  arXiv  Detail & Related papers  (2025-07-06T17:01:02Z)
• RealHarm: A Collection of Real-World Language Model Application Failures [1.2820953788225848]
  We introduce RealHarm, a dataset of annotated problematic interactions with AI agents.<n>We analyze harms, causes, and hazards specifically from the deployer's perspective.<n>We evaluate state-of-the-art guardrails and content moderation systems to probe whether such systems would have prevented the incidents.
  arXiv  Detail & Related papers  (2025-04-14T14:44:41Z)
• Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors [24.954755569786396]
  AI-text detection has emerged to distinguish between human and machine-generated content. Recent research indicates that these detection systems often lack robustness and struggle to effectively differentiate perturbed texts. Our work simulates real-world scenarios in both informal and professional writing, exploring the out-of-the-box performance of current detectors.
  arXiv  Detail & Related papers  (2024-06-13T08:37:01Z)
• Physical Adversarial Attack meets Computer Vision: A Decade Survey [55.38113802311365]
  This paper presents a comprehensive overview of physical adversarial attacks. We take the first step to systematically evaluate the performance of physical adversarial attacks. Our proposed evaluation metric, hiPAA, comprises six perspectives.
  arXiv  Detail & Related papers  (2022-09-30T01:59:53Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Adversarial vs behavioural-based defensive AI with joint, continual and active learning: automated evaluation of robustness to deception, poisoning and concept drift [62.997667081978825]
  Recent advancements in Artificial Intelligence (AI) have brought new capabilities to behavioural analysis (UEBA) for cyber-security. In this paper, we present a solution to effectively mitigate this attack by improving the detection process and efficiently leveraging human expertise.
  arXiv  Detail & Related papers  (2020-01-13T13:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.