The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems

• URL: http://arxiv.org/abs/2606.13079v1
• Date: Thu, 11 Jun 2026 09:02:14 GMT
• Title: The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems
• Abstract summary: A growing body of work has sought to assess the autonomous penetration capabilities of AI systems.<n>We construct a new autonomous penetration evaluation framework consisting of two components: target servers and agent scaffolding.<n>We find that current models achieve penetration success rates ranging from 10.7% to 69.3%.
• Score: 21.83197937022436
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Nowadays, the autonomous execution of cyberattacks capable of causing substantial real-world harm is widely regarded as one of the critical red lines that frontier AI systems must not cross. Within this broader red-line scenario, autonomous penetration represents a core enabling capability and subtask: the ability of LLM-powered AI systems to independently conduct adversarial operations against a target server without human intervention, identify and exploit vulnerabilities, and obtain unauthorized access or control. A growing body of work has sought to assess the autonomous penetration capabilities of AI systems. However, existing evaluations often employ opaque methodologies, rely on unrealistic or overly simplified penetration-testing scenarios, or provide LLMs with excessive prior knowledge and task-specific guidance, and cannot accurately capture the extent to which modern AI systems can autonomously perform this core capability within broader high-impact cyberattack scenarios. To address these limitations, we construct a new autonomous penetration evaluation framework consisting of two components: target servers and agent scaffolding. Specifically, on the target-server side, we design two levels of target environments based on the number of secure services without known vulnerabilities deployed alongside a vulnerable service: Tier~1 (one secure service) and Tier~2 (three secure services), resulting in a total of 300 target servers. Meanwhile, the agent scaffolding adopts a general-purpose agent architecture equipped with a set of general-purpose cybersecurity tools, without any target-specific prior knowledge. We evaluate 19 open-weight and proprietary LLMs, and find that current models achieve penetration success rates ranging from 10.7% to 69.3%. Moreover, we observe that autonomous penetration capability continues to improve alongside advances in overall model capability.

Related papers

• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild [49.44069582581181]
  CyberFactory transforms public vulnerability artifacts, including CVEs from the wild, into executable and verifiable task instances.<n>It uses a reusable vulnerability-analysis skill to guide the teacher through source inspection, problem solving with domain prior, and evidence-based validation.<n>On CyberGym, modelname reaches 52.4% Pass@1 under a one-hour budget, improving over its Qwen3.5 base model by +22.8 points.
  arXiv  Detail & Related papers  (2026-08-24T12:30:10Z)
• LLM-Centric Agentic AI for UAV Swarms: Architecture, Enabling Technologies, and Open Problems [4.4579151410787246]
  Uncrewed Aerial Vehicle (UAV) swarms have significant potential for applications such as Search and Rescue (SAR) and environmental monitoring.<n>Agentic Artificial Intelligence (AI) represents a shift from standalone Large Language Model (LLM) toward closed-loop cognitive architectures.<n>This paper proposes LLM-Centric Agentic AI for UAV Swarms (LAUS) and reviews key enabling technologies.
  arXiv  Detail & Related papers  (2026-07-05T13:08:06Z)
• Running the Gauntlet: Re-evaluating the Capabilities of Agents Beyond Familiar Environments [67.2234316079859]
  GauntletBench is a web-based benchmark for evaluating agent generalisation in challenging scenarios.<n>It focuses on three underexplored capabilities (temporal perception, graphical understanding, and 3D reasoning) across five less-covered professional applications.<n>Our empirical results reveal that frontier agentic systems remain far from achieving human-level performance.
  arXiv  Detail & Related papers  (2026-06-12T12:32:24Z)
• AgentCanary: A Security Evaluation Framework for Autonomous AI Agents in Real Executable Environments [32.982317538632806]
  We propose AgentCanary, a comprehensive security evaluation framework for autonomous AI agents.<n>AgentCanary provides a systematic solution along three contributions.<n>We evaluate a broad set of frontier models on AgentCanary against multiple established adversarial attack methods.
  arXiv  Detail & Related papers  (2026-06-09T06:55:37Z)
• CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities [66.48570184296857]
  CyberGym-E2E is a large-scale and realistic end-to-end cybersecurity benchmark.<n>It comprehensively evaluates AI agents' abilities across the full lifecycle of vulnerability discovery, PoC generation, and patch generation.<n>The benchmark consists of 920 real-world vulnerabilities across 139 different open-source projects.
  arXiv  Detail & Related papers  (2026-06-03T05:06:37Z)
• DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents [121.77550256034]
  DecodingTrust-Agent Platform (DTap) is a controllable and interactive red-teaming platform for AI agents.<n>DTap-Red is the first autonomous red-teaming agent that explores diverse injection vectors and autonomously discovers effective attack strategies.<n>Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models.
  arXiv  Detail & Related papers  (2026-05-06T11:59:48Z)
• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition [101.86739402748995]
  We run the largest public red-teaming competition to date, targeting 22 frontier AI agents across 44 realistic deployment scenarios.<n>We build the Agent Red Teaming benchmark and evaluate it across 19 state-of-the-art models.<n>Our findings highlight critical and persistent vulnerabilities in today's AI agents.
  arXiv  Detail & Related papers  (2025-07-28T05:13:04Z)
• Threat Modeling for AI: The Case for an Asset-Centric Approach [0.23408308015481666]
  AI systems now able to autonomously execute code, interact with external systems, and operate without human oversight.<n>With AI systems now able to autonomously execute code, interact with external systems, and operate without human oversight, traditional security approaches fall short.<n>This paper introduces an asset-centric methodology for threat modeling AI systems.
  arXiv  Detail & Related papers  (2025-05-08T18:57:08Z)
• Guardians of the Agentic System: Preventing Many Shots Jailbreak with Agentic System [0.8136541584281987]
  This work uses three examination methods to detect rogue agents through a Reverse Turing Test and analyze deceptive alignment through multi-agent simulations.<n>We develop an anti-jailbreaking system by testing it with GEMINI 1.5 pro and llama-3.3-70B, deepseek r1 models.<n>The detection capabilities are strong such as 94% accuracy for GEMINI 1.5 pro yet the system suffers persistent vulnerabilities when under long attacks.
  arXiv  Detail & Related papers  (2025-02-23T23:35:15Z)
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics [68.36528819227641]
  This paper systematically evaluates the robustness of Vision-Language-Action (VLA) models.<n>We introduce two untargeted attack objectives that leverage spatial foundations to destabilize robotic actions, and a targeted attack objective that manipulates the robotic trajectory.<n>We design an adversarial patch generation approach that places a small, colorful patch within the camera's view, effectively executing the attack in both digital and physical environments.
  arXiv  Detail & Related papers  (2024-11-18T01:52:20Z)
• Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems [27.316115171846953]
  Large Language Models (LLMs) have shown significant promise in real-world decision-making tasks for embodied AI.<n>LLMs are fine-tuned to leverage their inherent common sense and reasoning abilities while being tailored to specific applications.<n>This fine-tuning process introduces considerable safety and security vulnerabilities, especially in safety-critical cyber-physical systems.
  arXiv  Detail & Related papers  (2024-05-27T17:59:43Z)
• On the Vulnerability of LLM/VLM-Controlled Robotics [54.57914943017522]
  We highlight vulnerabilities in robotic systems integrating large language models (LLMs) and vision-language models (VLMs) due to input modality sensitivities.<n>Our results show that simple input perturbations reduce task execution success rates by 22.2% and 14.6% in two representative LLM/VLM-controlled robotic systems.
  arXiv  Detail & Related papers  (2024-02-15T22:01:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.