A Large Language Model Approach to Generating Bypass Rules for Malware Evasion in Analysis Sandbox

• URL: http://arxiv.org/abs/2605.21821v1
• Date: Wed, 20 May 2026 23:43:02 GMT
• Title: A Large Language Model Approach to Generating Bypass Rules for Malware Evasion in Analysis Sandbox
• Abstract summary: Existing approaches rely on manually crafted bypass rules that require deep reverse engineering of each evasion mechanism.<n>We propose ABLE, which analyzes execution traces from malware terminated due to potentially evasive behavior.<n>ABLE achieves a 79% bypass success rate, with iterative refinement contributing 29.5% of successful cases.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sandbox evasion remains a critical challenge for automated malware analysis, as modern malware employs environment checks to detect analysis platforms and suppress malicious behavior. Existing approaches rely on manually crafted bypass rules that require deep reverse engineering of each evasion mechanism -an approach that cannot scale against rapidly evolving evasion techniques. In this paper, we leverage large language models (LLMs) to automatically generate YARA rules that bypass evasion checks in sandbox environments. We propose ABLE, which analyzes execution traces from malware terminated due to potentially evasive behavior and employs multiple reasoning strategies to generate targeted bypass rules. To address syntactic errors and improve the efficacy of the bypass rules in the LLM outputs, we introduce an auto-sanitization pipeline and feedback-driven iterative refinement. We evaluate ABLE on 334 real-world malware samples across four open-weight LLMs. ABLE achieves a 79% bypass success rate, with iterative refinement contributing 29.5% of successful cases. Compared to existing analysis platforms, ABLE identifies 47% more malware family classifications and exposes previously hidden behaviors.

Related papers

• SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs [35.45887691929597]
  Large language models (LLMs) have shown promise in security analysis, but their ability to recover IOCs from malicious scripts remains underexplored.<n>We present SCRIPTIOC-BENCH, a benchmark for measuring static IOC extraction capability on real-world scripts.<n>We evaluate a broad range of proprietary and open-weight LLMs and show that IOC recovery without execution remains challenging across model scales.
  arXiv  Detail & Related papers  (2026-09-05T15:47:52Z)
• Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics [50.36375380196006]
  Jailbreak prompts can bypass alignment guardrails in large language models.<n>Prior detection approaches largely rely on a fixed metric space.<n>We show this assumption breaks under (i) pseudo-malicious prompts that are benign by intent but contain safety-related keywords.<n>We present Manifold Trajectory Kinetics (MTK), which treats an LLM as a kinetic system transforming inputs into outputs.
  arXiv  Detail & Related papers  (2026-06-05T14:49:26Z)
• Automatically Attacking Software Reverse Engineering AI Agents [0.3277163122167433]
  Software tools for reverse engineering executable binary files, such as Ghidra, enable malware analysts to safely conduct robust static analysis without having access to original source code.<n>This paper presents an adversarial technique using genetic algorithm-based prompt generation, a modification of an adversarial attack known as AutoDAN.
  arXiv  Detail & Related papers  (2026-05-28T23:58:25Z)
• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search [58.8834056209347]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs.<n>We introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base.
  arXiv  Detail & Related papers  (2025-12-01T07:05:23Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias [3.178301843099705]
  Large Language Models (LLMs) are increasingly trusted to perform automated code review and static analysis at scale.<n>This paper identifies and exploit a critical vulnerability in LLM-based code analysis.<n>We develop a fully automated, black-box algorithm that discovers and injects Familiar Pattern Attack (FPA) into target code.
  arXiv  Detail & Related papers  (2025-08-24T13:42:48Z)
• Enhanced Consistency Bi-directional GAN(CBiGAN) for Malware Anomaly Detection [0.25163931116642785]
  This paper introduces the application of the CBiGAN in the domain of malware anomaly detection.<n>We utilize several datasets including both portable executable (PE) files as well as Object Linking and Embedding (OLE) files.<n>We then evaluated our model against a diverse set of both PE and OLE files, including self-collected malicious executables from 214 malware families.
  arXiv  Detail & Related papers  (2025-06-09T02:43:25Z)
• Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities [49.09703018511403]
  Evaluations of large language model (LLM) risks and capabilities are increasingly being incorporated into AI risk management and governance frameworks.<n>Currently, most risk evaluations are conducted by designing inputs that elicit harmful behaviors from the system.<n>We propose evaluating LLMs with model tampering attacks which allow for modifications to latent activations or weights.
  arXiv  Detail & Related papers  (2025-02-03T18:59:16Z)
• Exploring LLMs for Malware Detection: Review, Framework Design, and Countermeasure Approaches [0.24578723416255752]
  The rising use of Large Language Models to create and disseminate malware poses a significant cybersecurity challenge. This paper provides a comprehensive overview of LLMs and their role in malware detection from diverse sources. We examine five specific applications of LLMs: Malware honeypots, identification of text-based threats, code analysis for detecting malicious intent, trend analysis of malware, and detection of non-standard disguised malware.
  arXiv  Detail & Related papers  (2024-09-11T19:33:44Z)
• DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness [58.23214712926585]
  We develop a certified defense, DRSM (De-Randomized Smoothed MalConv), by redesigning the de-randomized smoothing technique for the domain of malware detection. Specifically, we propose a window ablation scheme to provably limit the impact of adversarial bytes while maximally preserving local structures of the executables. We are the first to offer certified robustness in the realm of static detection of malware executables.
  arXiv  Detail & Related papers  (2023-03-20T17:25:22Z)
• MERLIN -- Malware Evasion with Reinforcement LearnINg [26.500149465292246]
  We propose a method using reinforcement learning with DQN and REINFORCE algorithms to challenge two state-of-the-art malware detection engines. Our method combines several actions, modifying a Windows portable execution file without breaking its functionalities. We demonstrate that REINFORCE achieves very good evasion rates even on a commercial AV with limited available information.
  arXiv  Detail & Related papers  (2022-03-24T10:58:47Z)
• Towards an Automated Pipeline for Detecting and Classifying Malware through Machine Learning [0.0]
  We propose a malware taxonomic classification pipeline able to classify Windows Portable Executable files (PEs) Given an input PE sample, it is first classified as either malicious or benign. If malicious, the pipeline further analyzes it in order to establish its threat type, family, and behavior(s)
  arXiv  Detail & Related papers  (2021-06-10T10:07:50Z)
• Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection [67.53296659361598]
  adversarial EXEmples can bypass machine learning-based detection by perturbing relatively few input bytes. We develop a unifying framework that does not only encompass and generalize previous attacks against machine-learning models, but also includes three novel attacks. These attacks, named Full DOS, Extend and Shift, inject the adversarial payload by respectively manipulating the DOS header, extending it, and shifting the content of the first section.
  arXiv  Detail & Related papers  (2020-08-17T07:16:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.