Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interaction

• URL: http://arxiv.org/abs/2603.27127v1
• Date: Sat, 28 Mar 2026 04:34:09 GMT
• Title: Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Reflective Verification and Knowledge-augmented Interaction
• Abstract summary: This paper proposes Red-MIRROR, a novel multi-agent automated penetration testing system.<n>Red-MIRROR introduces a tightly coupled memory-reflection backbone to explicitly govern inter-agent reasoning.<n> Empirical evaluation on the XBOW benchmark and Vulhub CVEs shows that Red-MIRROR achieves performance comparable to state-of-the-art agents.
• Score: 0.42222053626544365
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Web applications remain the dominant attack surface in cybersecurity, where vulnerabilities such as SQL injection, XSS, and business logic flaws continue to cause significant data breaches. While penetration testing is effective for identifying these weaknesses, traditional manual approaches are time-consuming and heavily dependent on scarce expert knowledge. Recent Large Language Models (LLM)-based multi-agent systems have shown promise in automating penetration testing, yet they still suffer from critical limitations: over-reliance on parametric knowledge, fragmented session memory, and insufficient validation of attack payloads and responses. This paper proposes Red-MIRROR, a novel multi-agent automated penetration testing system that introduces a tightly coupled memory-reflection backbone to explicitly govern inter-agent reasoning. By synthesizing Retrieval-Augmented Generation (RAG) for external knowledge augmentation, a Shared Recurrent Memory Mechanism (SRMM) for persistent state management, and a Dual-Phase Reflection mechanism for adaptive validation, Red-MIRROR provides a robust solution for complex web exploitation. Empirical evaluation on the XBOW benchmark and Vulhub CVEs shows that Red-MIRROR achieves performance comparable to state-of-the-art agents on Vulhub scenarios, while demonstrating a clear advantage on the XBOW benchmark. On the XBOW benchmark, Red-MIRROR attains an overall success rate of 86.0 percent, outperforming PentestAgent (50.0 percent), AutoPT (46.0 percent), and the VulnBot baseline (6.0 percent). Furthermore, the system achieves a 93.99 percent subtask completion rate, indicating strong long-horizon reasoning and payload refinement capability. Finally, we discuss ethical implications and propose safeguards to mitigate misuse risks.

Related papers

• Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing [8.293465530619246]
  We propose a two-stage decoupled evaluation framework that separates exploit execution from reconnaissance.<n>We empirically evaluate five open-source penetration-testing agents, covering multiagent, monolithic, and graph-driven architectures.
  arXiv  Detail & Related papers  (2026-06-24T02:51:58Z)
• Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents [93.19140872946842]
  Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments.<n>This makes them vulnerable to prompt-injection attacks, in which seemingly benign content embeds adversarial instructions that manipulate agent behaviour.<n>We introduce textbfsysname, a benchmark to systematically categorize and attribute harm in real-world web agent systems.
  arXiv  Detail & Related papers  (2026-06-11T14:12:43Z)
• Beyond Explicit Refusals: Soft-Failure Attacks on Retrieval-Augmented Generation [25.27360087818357]
  Existing jamming attacks on Retrieval-Augmented Generation (RAG) systems induce explicit refusals or denial-of-service behaviors.<n>We propose Deceptive Evolutionary Jamming Attack (DEJA), an automated black-box attack framework that generates adversarial documents to trigger such soft failures.<n>Experiments show that DEJA consistently drives responses toward low-utility soft failures, achieving SASR above 79% while keeping hard-failure rates below 15%.
  arXiv  Detail & Related papers  (2026-04-20T12:33:52Z)
• MA-IDS: Multi-Agent RAG Framework for IoT Network Intrusion Detection with an Experience Library [0.0]
  We propose MA-IDS, a Multi-Agent Intrusion Detection System that combines Large Language Models with Retrieval Augmented Generation for reasoning-driven intrusion detection.<n> evaluated on NF-BoT-IoT and NF-ToN-IoT benchmark datasets.<n>Results are competitive with SVM while providing rule-level explanations for every classification decision.
  arXiv  Detail & Related papers  (2026-04-07T05:41:10Z)
• AWE: Adaptive Agents for Dynamic Web Penetration Testing [0.0]
  AWE is a memory-augmented multi-agent framework for autonomous web penetration testing.<n>It embeds structured, vulnerability-specific analysis pipelines within a lightweight LLM orchestration layer.<n>AWE achieves substantial gains on injection-class vulnerabilities.
  arXiv  Detail & Related papers  (2026-03-01T07:32:42Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• Automated Red-Teaming Framework for Large Language Model Security Assessment: A Comprehensive Attack Generation and Detection System [4.864011355064205]
  This paper introduces an automated red-teaming framework that generates, executes, and evaluates adversarial prompts to uncover security vulnerabilities in large language models (LLMs)<n>Our framework integrates meta-prompting-based attack synthesis, multi-modal vulnerability detection, and standardized evaluation protocols spanning six major threat categories.<n> Experiments on the GPT-OSS-20B model reveal 47 distinct vulnerabilities, including 21 high-severity and 12 novel attack patterns.
  arXiv  Detail & Related papers  (2025-12-21T19:12:44Z)
• MAVUL: Multi-Agent Vulnerability Detection via Contextual Reasoning and Interactive Refinement [9.377934769326416]
  MAVUL is a novel multi-agent vulnerability detection system that integrates contextual reasoning and interactive refinement.<n>We show MAVUL significantly outperforms existing multi-agent systems with over 62% higher pairwise accuracy and single-agent systems with over 600% higher average performance.
  arXiv  Detail & Related papers  (2025-09-30T22:21:43Z)
• D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models [62.83226685925107]
  Deceptive Reasoning Exposure Suite (D-REX) is a novel dataset designed to evaluate the discrepancy between a model's internal reasoning process and its final output.<n>Each sample in D-REX contains the adversarial system prompt, an end-user's test query, the model's seemingly innocuous response, and, crucially, the model's internal chain-of-thought.<n>We demonstrate that D-REX presents a significant challenge for existing models and safety mechanisms.
  arXiv  Detail & Related papers  (2025-09-22T15:59:40Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• AI Agentic Vulnerability Injection And Transformation with Optimized Reasoning [2.918225266151982]
  We present AVIATOR, the first AI-agentic vulnerability injection workflow.<n>It automatically injects realistic, category-specific vulnerabilities for high-fidelity, diverse, large-scale vulnerability dataset generation.<n>It combines semantic analysis, injection synthesis enhanced with LoRA-based fine-tuning and Retrieval-Augmented Generation, as well as post-injection validation via static analysis and LLM-based discriminators.
  arXiv  Detail & Related papers  (2025-08-28T14:59:39Z)
• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• Jailbreaking as a Reward Misspecification Problem [80.52431374743998]
  We propose a novel perspective that attributes this vulnerability to reward misspecification during the alignment process.<n>We introduce a metric ReGap to quantify the extent of reward misspecification and demonstrate its effectiveness.<n>We present ReMiss, a system for automated red teaming that generates adversarial prompts in a reward-misspecified space.
  arXiv  Detail & Related papers  (2024-06-20T15:12:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.