Defeat Devices in AI Systems

• URL: http://arxiv.org/abs/2606.28863v1
• Date: Sat, 27 Jun 2026 11:12:17 GMT
• Title: Defeat Devices in AI Systems
• Abstract summary: A defeat device is an engineering concept long established in vehicle-emissions law and brought to broad public attention by the 2015 Volkswagen emissions case.<n>We formalize this triadic test as a behavioral definition, organize documented cases along three axes (origin, trigger, swap mechanism), and propose Trigger-Axis-Aware Differential Probing (TADP) as a forensic detection protocol.<n>We characterize naturally-emerging defeat devices as potentially one of the harmful emerging phenomena that AI safety practice should monitor and test for systematically.
• Score: 8.970269049715933
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI systems increasingly exhibit behavior that differs systematically between evaluation and deployment contexts. Alignment faking, sandbagging, benchmark gaming, deceptive scheming, specification gaming, and trojans have each been documented separately, with each line of work characterizing one facet of what we argue is a single structural mechanism. We propose that this common mechanism is a defeat device, an engineering and regulatory concept long established in vehicle-emissions law and brought to broad public attention by the 2015 Volkswagen emissions case. A defeat device in an AI system has three necessary elements: a discriminator that detects evaluation context, a concealed swap that conditions behavior on detection, and a gap between eval-distribution and deployment-distribution performance on the stated evaluation criterion. We formalize this triadic test as a behavioral definition, organize documented cases along three taxonomic axes (origin, trigger, swap mechanism), propose Trigger-Axis-Aware Differential Probing (TADP) as a forensic detection protocol, and advance the claim that defeat devices can naturally emerge in current frontier AI systems without any operator engineering. We characterize naturally-emerging defeat devices as potentially one of the harmful emerging phenomena that AI safety practice should monitor and test for systematically. Implications for evaluation methodology, post-training pipeline design, interpretability research priorities, and AI governance follow.

Related papers

• AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation [48.55515767840701]
  "AI slop" is artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports.<n>This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage.
  arXiv  Detail & Related papers  (2026-08-26T11:48:38Z)
• Constraint-Driven Synthesis of Hyper Petri Nets [46.9047309711542]
  This paper addresses the modeling and synthesis of constrained robotic system behaviors using Petri nets (PNs)<n>It investigates how to construct models in which all observable system states satisfy given logical constraints while remaining consistent with executable transition semantics.<n>The proposed method introduces an explicit execution semantics over observable states, induced by admissible (atomic) firing sequences.
  arXiv  Detail & Related papers  (2026-07-24T07:57:19Z)
• Rethinking Penetration Testing for AI-Enabled Systems: From Resource Compromise to Behavioral Objective Violation [0.0]
  Penetration testing traditionally evaluates whether adversaries can exploit weaknesses in software, infrastructure, configurations, or operational controls to achieve security-relevant compromise.<n>This paper reframes penetration testing for AI-enabled systems as objective-driven behavioral evaluation.
  arXiv  Detail & Related papers  (2026-07-15T16:36:54Z)
• Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance [0.0]
  This paper contributes a seven-dimension profile distinguishing pure software-engineering, hybrid, and AI-native teams.<n>Because the object of study is framework adequacy rather than human behavior, the evaluation yields derived rather than observed coverage claims.
  arXiv  Detail & Related papers  (2026-07-01T19:31:02Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• Agentic AI for Cybersecurity: A Meta-Cognitive Architecture for Governable Autonomy [0.0]
  This paper argues that cybersecurity orchestration should be reconceptualized as an agentic, multi-agent cognitive system.<n>We introduce a conceptual framework in which heterogeneous AI agents responsible for detection, hypothesis formation, contextual interpretation, explanation, and governance are coordinated through an explicit meta-cognitive judgement function.<n>Our contribution is to make this cognitive structure architecturally explicit and governable by embedding meta-cognitive judgement as a first-class system function.
  arXiv  Detail & Related papers  (2026-02-12T12:52:49Z)
• Constructing Safety Cases for AI Systems: A Reusable Template Framework [10.44708664414503]
  Safety cases, structured arguments that a system is acceptably safe, are becoming central to the governance of AI systems.<n>Traditional safety-case practices from aviation or nuclear engineering rely on well-specified system boundaries, stable architectures, and known failure modes.<n>This study examines how safety cases are currently constructed for AI systems and why classical approaches fail to capture these dynamics.
  arXiv  Detail & Related papers  (2026-01-30T09:53:22Z)
• AI Deception: Risks, Dynamics, and Controls [153.71048309527225]
  This project provides a comprehensive and up-to-date overview of the AI deception field.<n>We identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception.<n>We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment.
  arXiv  Detail & Related papers  (2025-11-27T16:56:04Z)
• CANTXSec: A Deterministic Intrusion Detection and Prevention System for CAN Bus Monitoring ECU Activations [53.036288487863786]
  We propose CANTXSec, the first deterministic Intrusion Detection and Prevention system based on physical ECU activations.<n>It detects and prevents classical attacks in the CAN bus, while detecting advanced attacks that have been less investigated in the literature.<n>We prove the effectiveness of our solution on a physical testbed, where we achieve 100% detection accuracy in both classes of attacks while preventing 100% of FIAs.
  arXiv  Detail & Related papers  (2025-05-14T13:37:07Z)
• No Need to Know Physics: Resilience of Process-based Model-free Anomaly Detection for Industrial Control Systems [95.54151664013011]
  We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system. We analyze four anomaly detectors published at top security conferences.
  arXiv  Detail & Related papers  (2020-12-07T11:02:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.