Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks

• URL: http://arxiv.org/abs/2502.04227v2
• Date: Fri, 20 Jun 2025 12:02:54 GMT
• Title: Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks
• Authors: Andreas Happe, Jürgen Cito,
• Abstract summary: This paper investigates the feasibility and effectiveness of using Large Language Model (LLM)-driven autonomous systems to address challenges in real-world Microsoft Active Directory (AD) enterprise networks.<n>Our prototype, cochise, represents the first demonstration of a fully autonomous, LLM-driven framework capable of compromising accounts within a real-life Microsoft AD testbed (GOAD)<n>Key findings highlight their ability to dynamically adapt attack strategies, perform inter-context attacks, and generate scenario-specific attack parameters.
• Score: 3.11537581064266
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Penetration-testing, while critical for validating defenses and uncovering vulnerabilities, is often limited by high operational costs and the scarcity of human expertise. This paper investigates the feasibility and effectiveness of using Large Language Model (LLM)-driven autonomous systems to address these challenges in real-world Microsoft Active Directory (AD) enterprise networks. Our novel prototype, cochise, represents the first demonstration of a fully autonomous, LLM-driven framework capable of compromising accounts within a real-life Microsoft AD testbed (GOAD). The evaluation deliberately utilizes GOAD to capture the intricate interactions and sometimes nondeterministic outcomes of live network pen-testing, moving beyond the limitations of synthetic benchmarks. We perform our empirical evaluation using five LLMs, comparing reasoning to non-reasoning models as well as including open-weight models. Through comprehensive quantitative and qualitative analysis, incorporating insights from cybersecurity experts, we demonstrate that autonomous LLMs can effectively conduct Assumed Breach simulations. Key findings highlight their ability to dynamically adapt attack strategies, perform inter-context attacks, and generate scenario-specific attack parameters. Cochise also exhibits robust self-correction mechanisms, automatically installing missing tools and rectifying invalid command generations. Critically, we find that the associated costs are competitive with those incurred by professional pen-testers, suggesting a path toward democratizing access to essential security testing for organizations with budgetary constraints. However, our research also illuminates existing limitations, including instances of LLM ``going down rabbit holes'', challenges in comprehensive information transfer between planning and execution modules, and critical safety concerns that necessitate human oversight.

Related papers

• White-Basilisk: A Hybrid Model for Code Vulnerability Detection [50.49233187721795]
  We introduce White-Basilisk, a novel approach to vulnerability detection that demonstrates superior performance.<n>White-Basilisk achieves results in vulnerability detection tasks with a parameter count of only 200M.<n>This research establishes new benchmarks in code security and provides empirical evidence that compact, efficiently designed models can outperform larger counterparts in specialized tasks.
  arXiv  Detail & Related papers  (2025-07-11T12:39:25Z)
• On the Surprising Efficacy of LLMs for Penetration-Testing [3.11537581064266]
  The paper thoroughly reviews the evolution of Large Language Models (LLMs) in penetration testing.<n>It showcases their application across various offensive security tasks and covering broader phases of the cyber kill chain.<n>The paper identifies and discusses significant obstacles impeding wider adoption and safe deployment.
  arXiv  Detail & Related papers  (2025-07-01T15:01:18Z)
• Expert-in-the-Loop Systems with Cross-Domain and In-Domain Few-Shot Learning for Software Vulnerability Detection [38.083049237330826]
  This study explores the use of Large Language Models (LLMs) in software vulnerability assessment by simulating the identification of Python code with known Common Weaknessions (CWEs)<n>Our results indicate that while zero-shot prompting performs poorly, few-shot prompting significantly enhances classification performance.<n> challenges such as model reliability, interpretability, and adversarial robustness remain critical areas for future research.
  arXiv  Detail & Related papers  (2025-06-11T18:43:51Z)
• LLMpatronous: Harnessing the Power of LLMs For Vulnerability Detection [0.0]
  Large Language Models (LLMs) for vulnerability detection presents unique challenges.<n>Previous attempts employing machine learning models for vulnerability detection have proven ineffective.<n>We propose a robust AI-driven approach focused on mitigating these limitations.
  arXiv  Detail & Related papers  (2025-04-25T15:30:40Z)
• Large Language Models powered Network Attack Detection: Architecture, Opportunities and Case Study [26.966976709473226]
  Large Language Models (LLMs) are trained on a vast corpus of text. This has opened up a new door for network threat detection. We present our design on LLM-powered DDoS detection as a case study.
  arXiv  Detail & Related papers  (2025-03-24T09:40:46Z)
• How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities [62.474732677086855]
  Large language model (LLM) routing has emerged as a crucial strategy for balancing computational costs with performance.<n>We propose the DSC benchmark: Diverse, Simple, and Categorized, an evaluation framework that categorizes router performance across a broad spectrum of query types.
  arXiv  Detail & Related papers  (2025-03-20T19:52:30Z)
• Construction and Evaluation of LLM-based agents for Semi-Autonomous penetration testing [0.0]
  High-performance large language models (LLMs) have advanced across various domains.<n>In highly specialized fields such as cybersecurity, full autonomy remains a challenge.<n>We propose a system that semi-autonomously executes complex cybersecurity by employing multiple LLMs modules.
  arXiv  Detail & Related papers  (2025-02-21T15:02:39Z)
• OCCULT: Evaluating Large Language Models for Offensive Cyber Operation Capabilities [0.0]
  We demonstrate a new approach to assessing AI's progress towards enabling and scaling real-world offensive cyber operations.<n>We detail OCCULT, a lightweight operational evaluation framework that allows cyber security experts to contribute to rigorous and repeatable measurement.<n>We find that there has been significant recent advancement in the risks of AI being used to scale realistic cyber threats.
  arXiv  Detail & Related papers  (2025-02-18T19:33:14Z)
• Scaling Autonomous Agents via Automatic Reward Modeling And Planning [52.39395405893965]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of tasks. However, they still struggle with problems requiring multi-step decision-making and environmental feedback. We propose a framework that can automatically learn a reward model from the environment without human annotations.
  arXiv  Detail & Related papers  (2025-02-17T18:49:25Z)
• Risk-Aware Driving Scenario Analysis with Large Language Models [7.093690352605479]
  Large Language Models (LLMs) can capture nuanced contextual relationships, reasoning, and complex problem-solving.<n>This paper proposes a novel framework that leverages LLMs for risk-aware analysis of generated driving scenarios.
  arXiv  Detail & Related papers  (2025-02-04T09:19:13Z)
• Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving [65.61999354218628]
  We take the first step toward designing black-box adversarial attacks specifically targeting vision-language models (VLMs) in autonomous driving systems.<n>We propose Cascading Adversarial Disruption (CAD), which targets low-level reasoning breakdown by generating and injecting semantics.<n>We present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios.
  arXiv  Detail & Related papers  (2025-01-23T11:10:02Z)
• AutoPT: How Far Are We from the End2End Automated Web Penetration Testing? [54.65079443902714]
  We introduce AutoPT, an automated penetration testing agent based on the principle of PSM driven by LLMs. Our results show that AutoPT outperforms the baseline framework ReAct on the GPT-4o mini model.
  arXiv  Detail & Related papers  (2024-11-02T13:24:30Z)
• Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization [62.16747639440893]
  Large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering.<n>Our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions.
  arXiv  Detail & Related papers  (2024-08-07T08:43:32Z)
• ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning [74.58666091522198]
  We present a framework for intuitive robot programming by non-experts. We leverage natural language prompts and contextual information from the Robot Operating System (ROS) Our system integrates large language models (LLMs), enabling non-experts to articulate task requirements to the system through a chat interface.
  arXiv  Detail & Related papers  (2024-06-28T08:28:38Z)
• Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents [101.17919953243107]
  GovSim is a generative simulation platform designed to study strategic interactions and cooperative decision-making in large language models (LLMs)<n>We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in GovSim, with the highest survival rate below 54%.<n>We show that agents that leverage "Universalization"-based reasoning, a theory of moral thinking, are able to achieve significantly better sustainability.
  arXiv  Detail & Related papers  (2024-04-25T15:59:16Z)
• Can LLMs Understand Computer Networks? Towards a Virtual System Administrator [15.469010487781931]
  This paper is the first to conduct an exhaustive study on Large Language Models' comprehension of computer networks. We evaluate our framework on multiple computer networks employing proprietary (e.g., GPT4) and open-source (e.g., Llama2) models.
  arXiv  Detail & Related papers  (2024-04-19T07:41:54Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)
• RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models [46.476439550746136]
  Large language model (LLM) applications in cloud root cause analysis (RCA) have been actively explored recently. We present RCAgent, a tool-augmented LLM autonomous agent framework for practical and privacy-aware industrial RCA usage. Running on an internally deployed model rather than GPT families, RCAgent is capable of free-form data collection and comprehensive analysis with tools.
  arXiv  Detail & Related papers  (2023-10-25T03:53:31Z)
• LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks [0.0]
  We explore the intersection of Language Models (LLMs) and penetration testing. We introduce a fully automated privilege-escalation tool for evaluating the efficacy of LLMs for (ethical) hacking. We analyze the impact of different context sizes, in-context learning, optional high-level mechanisms, and memory management techniques.
  arXiv  Detail & Related papers  (2023-10-17T17:15:41Z)
• Getting pwn'd by AI: Penetration Testing with Large Language Models [0.0]
  This paper explores the potential usage of large-language models, such as GPT3.5, to augment penetration testers with AI sparring partners. We explore the feasibility of supplementing penetration testers with AI models for two distinct use cases: high-level task planning for security testing assignments and low-level vulnerability hunting within a vulnerable virtual machine.
  arXiv  Detail & Related papers  (2023-07-24T19:59:22Z)
• Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond [171.07853346630057]
  Linear relaxation based perturbation analysis (LiRPA) for neural networks has become a core component in robustness verification and certified defense. We develop an automatic framework to enable perturbation analysis on any neural network structures. We demonstrate LiRPA based certified defense on Tiny ImageNet and Downscaled ImageNet.
  arXiv  Detail & Related papers  (2020-02-28T18:47:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.