The End of Trust: How Agentic AI Breaks Security Assumptions

• URL: http://arxiv.org/abs/2605.16436v1
• Date: Thu, 14 May 2026 21:30:06 GMT
• Title: The End of Trust: How Agentic AI Breaks Security Assumptions
• Abstract summary: We introduce the Infinite Impostor, an attack model in which an autonomous agent interposes itself between two parties who already trust each other.<n>We propose a suspect-by-default paradigm that shifts security from authenticating actors to evaluating actions.
• Score: 45.13166202592558
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: For decades, the security of digital interaction has rested on an unacknowledged economic constraint. Attackers faced a tradeoff between the fidelity of a deception and the scale at which it could be deployed. Convincing impersonation required sustained human effort and was confined to a narrow set of high-value targets, while mass-market attacks sacrificed plausibility for reach. Detection systems, verification mechanisms, and user awareness training have all been implicitly calibrated to the artifacts of cheap deception that this tradeoff produced. Agentic AI collapses the tradeoff, allowing high-fidelity, individually tailored deception to be produced at mass-market scale. We argue that this shift exhausts a security paradigm rather than merely intensifying the threat landscape. We introduce the Infinite Impostor, an attack model in which an autonomous agent interposes itself between two parties who already trust each other, hijacking an existing relationship rather than building a new one from scratch. Detection-oriented defenses share an assumption that generative progress is eliminating, that synthetic outputs are distinguishable from authentic ones. We propose a suspect-by-default paradigm that shifts security from authenticating actors to evaluating actions, and examine the governance tensions that arise when platforms become the regulatory substrate of digital interaction.

Related papers

• TrojanWorld: Backdooring World-Model Agents via Imagination Steering [65.38754125569848]
  TrojanWorld is a backdoor framework for world-model agents that induces attacker-specified behavior by steering internal imagination.<n>Under trigger activation, TrojanWorld achieves a target-action deviation as low as 0.026 while retaining at least 98.8% of the corresponding clean performance.
  arXiv  Detail & Related papers  (2026-09-07T05:14:10Z)
• Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle [65.6642776933861]
  This survey examines the protective paradigm that has grown around this intervention point.<n>We find that most protections are still validated mainly against static or weakly adaptive adversaries.<n>We close by consolidating cross-stage countermeasures and open problems for robust, composable, and deployable owner-side protection.
  arXiv  Detail & Related papers  (2026-08-05T00:46:50Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• Breaking the Secret: Economic Interventions for Combating Collusion in Embodied Multi-Agent Systems [22.697901881084068]
  Collusion among autonomous agents poses a critical security threat in embodied multi-agent systems.<n>We propose an incentive intervention approach that mitigates collusion by reshaping agents' payoff structures.<n> Experimental results show that our method effectively suppresses collusion by inducing defection, while preserving system efficiency.
  arXiv  Detail & Related papers  (2026-04-26T03:13:47Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networks [32.791374704820925]
  Malicious or compromised agents can exploit communication channels to propagate misinformation and manipulate collective outcomes.<n>We study how such manipulation can arise and spread by borrowing the Friedkin-Johnsen opinion formation model from social sciences.<n>We propose a new mechanism to mitigate threats by a trust-adaptive defense that dynamically adjusts inter-agent trust to limit adversarial influence.
  arXiv  Detail & Related papers  (2026-03-16T18:41:21Z)
• OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage [59.3826294523924]
  We investigate the security vulnerabilities of a popular multi-agent pattern known as the orchestrator setup.<n>We report the susceptibility of frontier models to different categories of attacks, finding that both reasoning and non-reasoning models are vulnerable.
  arXiv  Detail & Related papers  (2026-02-13T21:32:32Z)
• The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies [57.387081435669835]
  Multi-agent systems built from large language models offer a promising paradigm for scalable collective intelligence and self-evolution.<n>We show that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible.<n>We propose several solution directions to alleviate the identified safety concern.
  arXiv  Detail & Related papers  (2026-02-10T15:18:19Z)
• SoK: Trust-Authorization Mismatch in LLM Agent Interactions [16.633676842555044]
  Large Language Models (LLMs) are rapidly evolving into autonomous agents capable of interacting with the external world.<n>This paper provides a unifying formal lens for agent-interaction security.<n>We introduce a novel risk analysis model centered on the trust-authorization gap.
  arXiv  Detail & Related papers  (2025-12-07T16:41:02Z)
• SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detection [22.242243610133215]
  Malicious agents pose significant threats to the reliability and decision-making capabilities of Multi-Agent Systems (MAS) powered by Large Language Models (LLMs)<n>We propose SentinelNet, the first decentralized framework for proactively detecting and mitigating malicious behaviors in multi-agent collaboration.<n> Experiments on MAS benchmarks show SentinelNet achieves near-perfect detection of malicious agents, close to 100% within two debate rounds, and recovers 95% of system accuracy from compromised baselines.
  arXiv  Detail & Related papers  (2025-10-17T21:10:35Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.