RedShell: A Generative AI-Based Approach to Ethical Hacking

• URL: http://arxiv.org/abs/2604.11506v1
• Date: Mon, 13 Apr 2026 14:10:44 GMT
• Title: RedShell: A Generative AI-Based Approach to Ethical Hacking
• Abstract summary: We propose RedShell, a tool that allows ethical hackers to generate malicious code.<n>Our experiments demonstrate the strong capabilities of RedShell in generating syntactically valid, with fewer than 10% of the generated samples resulting in parse errors.<n>This work sheds light on the state-of-the-art research in the field of Generative AI applied to Pentesting, and also serves as a steppingstone for future advancements.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The application of Machine Learning techniques in code generation is now a common practice for most developers. Tools such as ChatGPT from OpenAI leverage the natural language processing capabilities of Large Language Models to generate machine code from natural language descriptions. In the cybersecurity field, red teams can also take advantage of generative models to build malicious code generators, providing more automation to Pentest audits. However, the application of Large Language Models in malicious code generation remains challenging due to the lack of data to train and evaluate offensive code generators. In this work, we propose RedShell, a tool that allows ethical hackers to generate malicious PowerShell code. We also introduce a ground truth dataset, combining publicly available code samples to fine-tune models in malicious PowerShell generation. Our experiments demonstrate the strong capabilities of RedShell in generating syntactically valid PowerShell, with fewer than 10% of the generated samples resulting in parse errors. Furthermore, our specialized model was able to produce samples that were semantically consistent with reference snippets, achieving a competitive performance on standard output similarity metrics such as Edit Distance and METEOR, with their mean similarity scores exceeding 50% and 40%, respectively. This work sheds light on the state-of-the-art research in the field of Generative AI applied to Pentesting, and also serves as a steppingstone for future advancements, highlighting the potential benefits these models hold within such controlled environments.

Related papers

• Towards Automated Pentesting with Large Language Models [0.0]
  Large Language Models (LLMs) are redefining offensive cybersecurity by allowing the generation of harmful machine code with minimal human intervention.<n>We present RedShell, a framework that generates malicious code targeting Microsoft Windows vulnerabilities.
  arXiv  Detail & Related papers  (2026-04-13T17:41:51Z)
• Lightweight Yet Secure: Secure Scripting Language Generation via Lightweight LLMs [14.355160053196824]
  Large language models (LLMs) have demonstrated strong capabilities in code generation, vulnerability detection, and automated repair for languages like Python and JavaScript.<n>We present SecGenEval-PS, a benchmark designed to evaluate LLMs on secure scripting generation, security analysis, and automated repair.<n>We propose PSSec, a framework that combines data synthesis with fine-tuning to enhance model security capabilities.
  arXiv  Detail & Related papers  (2026-01-10T04:00:56Z)
• ATGen: Adversarial Reinforcement Learning for Test Case Generation [78.48498301767079]
  Large Language Models (LLMs) excel at code generation, yet their outputs often contain subtle bugs.<n>Existing test generation methods rely on static datasets.<n>We introduce ATGen, a framework that trains a test case generator via adversarial reinforcement learning.
  arXiv  Detail & Related papers  (2025-10-16T12:49:25Z)
• Ensuring Functional Correctness of Large Code Models with Selective Generation [13.517414451760635]
  hallucination of code generation models hinders their applicability to systems requiring higher safety standards.<n>We propose emphselective code generator that abstains from uncertain generations based on functional correctness evaluated by generated unit tests.<n>We demonstrate the efficacy of our method along with the controllability of code hallucination and reasonable selection efficiency.
  arXiv  Detail & Related papers  (2025-05-19T06:29:16Z)
• CodeRAG-Bench: Can Retrieval Augment Code Generation? [78.37076502395699]
  We conduct a systematic, large-scale analysis of code generation using retrieval-augmented generation.<n>We first curate a comprehensive evaluation benchmark, CodeRAG-Bench, encompassing three categories of code generation tasks.<n>We examine top-performing models on CodeRAG-Bench by providing contexts retrieved from one or multiple sources.
  arXiv  Detail & Related papers  (2024-06-20T16:59:52Z)
• The Power of Words: Generating PowerShell Attacks from Natural Language [4.593752628215474]
  This work explores an uncharted domain in AI code generation using Machine Translation (NMT) We present an extensive evaluation of state-of-the-art NMT models and analyze the generated code both statically and dynamically.
  arXiv  Detail & Related papers  (2024-04-19T13:54:34Z)
• Unveiling the potential of large language models in generating semantic and cross-language clones [8.791710193028905]
  OpenAI's GPT model has potential in such clone generation as GPT is used for text generation. In the realm of semantic clones, GPT-3 attains an impressive accuracy of 62.14% and 0.55 BLEU score, achieved through few-shot prompt engineering.
  arXiv  Detail & Related papers  (2023-09-12T17:40:49Z)
• CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
  Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
  arXiv  Detail & Related papers  (2023-02-08T11:54:07Z)
• ReCode: Robustness Evaluation of Code Generation Models [90.10436771217243]
  We propose ReCode, a comprehensive robustness evaluation benchmark for code generation models. We customize over 30 transformations specifically for code on docstrings, function and variable names, code syntax, and code format. With human annotators, we verified that over 90% of the perturbed prompts do not alter the semantic meaning of the original prompt.
  arXiv  Detail & Related papers  (2022-12-20T14:11:31Z)
• Interactive Code Generation via Test-Driven User-Intent Formalization [60.90035204567797]
  Large language models (LLMs) produce code from informal natural language (NL) intent. It is hard to define a notion of correctness since natural language can be ambiguous and lacks a formal semantics. We describe a language-agnostic abstract algorithm and a concrete implementation TiCoder.
  arXiv  Detail & Related papers  (2022-08-11T17:41:08Z)
• Measuring Coding Challenge Competence With APPS [54.22600767666257]
  We introduce APPS, a benchmark for code generation. Our benchmark includes 10,000 problems, which range from having simple one-line solutions to being substantial algorithmic challenges. Recent models such as GPT-Neo can pass approximately 15% of the test cases of introductory problems.
  arXiv  Detail & Related papers  (2021-05-20T17:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.