Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing

• URL: http://arxiv.org/abs/2609.07344v1
• Date: Mon, 07 Sep 2026 11:09:01 GMT
• Title: Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing
• Abstract summary: Intentest is an intent-graph-guided automated penetration testing agent.<n>It externalizes long-horizon state from the LLM's context window onto a persistent fact-intent directed acyclic graph.<n>It achieves an overall success rate of 88.2% and a success rate of 75.0% on hard tasks.
• Score: 17.809156797437208
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) based agents are increasingly applied to cybersecurity tasks such as vulnerability discovery and automated penetration testing. On long-horizon security tasks, however, such agents remain limited by context forgetting and intent drift: early critical facts and causal reasoning chains are lost over extended interactions, and the agent falls into aimless, repetitive exploration. This paper proposes Intentest, an intent-graph-guided automated penetration testing agent that externalizes long-horizon state from the LLM's context window onto a persistent fact-intent directed acyclic graph (DAG), thereby substantially reducing invalid transitions. We evaluate Intentest on automated penetration testing of web applications, a representative long-tail task in cybersecurity. In the DAG, verified network states are stored as immutable fact nodes, and exploration directions are constrained as intent edges bounded by predecessor facts. The system adopts a three-layer architecture, in which the fact-intent mapping layer maintains the global state, the task scheduling and allocation layer ensures execution stability through two-phase degradation recovery and multi-dimensional adaptive load balancing, and the intent retrieval and prediction layer provides tactical priors through a top-down five-stage filtering algorithm. On a benchmark of real CTF challenges covering more than ten vulnerability types across three difficulty levels, Intentest achieves an overall success rate of 88.2% and a success rate of 75.0% on hard tasks, improving over the baseline by approximately 44 and 50 percentage points. Ablation experiments further show that the intent retrieval and prediction reduce the average number of rounds on successful medium and hard tasks by about 33% and 48%, respectively, without changing the set of solvable tasks.

Related papers

• SherAgent: Scaling Attack Investigation in the Wild via LLM-Empowered Iterative Query-Filter Backtracking [32.566471484012276]
  Provenance-based attack investigation enables viable automation by standardizing data and query logic.<n>We collaborate with the SOC of a major Internet corporation serving billions of users to develop SherAgent.<n>SherAgent operates with remarkable efficiency, incurring under $0.10 in API costs and requiring less than 4 minutes per investigation.
  arXiv  Detail & Related papers  (2026-07-10T08:11:10Z)
• SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation [3.511540973608371]
  SCATE is a framework for adaptive, automated supervision of coding agents.<n>It learns to select the most promising testing actions based on the current coverage and class testability metrics.<n>It achieves 32.3% higher line coverage and 30.9% higher branch coverage than the agent-only baseline.
  arXiv  Detail & Related papers  (2026-07-09T23:13:46Z)
• Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing [8.293465530619246]
  We propose a two-stage decoupled evaluation framework that separates exploit execution from reconnaissance.<n>We empirically evaluate five open-source penetration-testing agents, covering multiagent, monolithic, and graph-driven architectures.
  arXiv  Detail & Related papers  (2026-06-24T02:51:58Z)
• GTA: Generating Long-Horizon Tasks for Web Agents at Scale [82.43869456830664]
  We introduce a scalable framework, GTA, that integrates crawling, retrieval-based seeding, in-context generation, and automated quality control.<n>We instantiate the pipeline on over 50 websites covering e-commerce, government, forums, and news, with multilingual and multi-hop coverage.<n>Our contributions are three-fold: (i) formalizing multi-hop web-agent task generation, (ii) proposing an efficient and validated pipeline for automatic data creation, and (iii) releasing a dynamic benchmark with reproducible evaluation.
  arXiv  Detail & Related papers  (2026-05-28T01:05:50Z)
• From Detection to Response: A Deep Learning and Retrieval-Augmented Generation Framework for Network Intrusion Mitigation [0.7136933021609079]
  This paper presents a unified, end-to-end framework that closes the gap between threat detection and actionable response.<n>The system operates in two tightly coupled stages. First, an ensemble of three independently trained binary Deep Neural Networks (DNNs) classifies network traffic flows as Benign, Denial of Service (DoS) or Distributed Denial of Service (DDoS)<n>Second, a Retrieval-Augmented Generation (RAG) pipeline constructs explanation-aware prompts from the top-5 anomalous features.
  arXiv  Detail & Related papers  (2026-05-18T07:17:55Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• What Makes a Good LLM Agent for Real-world Penetration Testing? [37.56537537883771]
  We analyze 28 LLM-based penetration testing systems and evaluate five representative implementations across three benchmarks of increasing complexity.<n>We show that Type B failures share a root cause that is largely invariant to the underlying LLM: agents lack real-time task difficulty estimation.<n>We present Excalibur, a penetration testing agent that couples strong tooling with difficulty-aware planning.
  arXiv  Detail & Related papers  (2026-02-19T18:42:40Z)
• LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces [65.11019654023978]
  LongCLI-Bench is a benchmark designed to evaluate agentic capabilities across long-horizon, realistic tasks.<n>We curated 20 high-quality, long-horizon tasks from over 1,000 computer science assignments and real-world tasks.<n>Experiments reveal that even state-of-the-art agents achieve pass rates below 20% in LongCLI-Bench.
  arXiv  Detail & Related papers  (2026-02-15T23:12:57Z)
• It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents [52.81924177620322]
  Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking.<n>Their reliance on dynamic web content makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task.<n>We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), an evaluation for studying how persuasion techniques misguide autonomous web agents on realistic tasks.
  arXiv  Detail & Related papers  (2025-12-29T01:09:10Z)
• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.