SoK: An Essential Guide For Using Malware Sandboxes In Security Applications: Challenges, Pitfalls, and Lessons Learned

• URL: http://arxiv.org/abs/2403.16304v1
• Date: Sun, 24 Mar 2024 21:41:41 GMT
• Title: SoK: An Essential Guide For Using Malware Sandboxes In Security Applications: Challenges, Pitfalls, and Lessons Learned
• Authors: Omar Alrawi, Miuyin Yong Wong, Athanasios Avgetidis, Kevin Valakuzhy, Boladji Vinny Adjibi, Konstantinos Karakatsanis, Mustaque Ahamad, Doug Blough, Fabian Monrose, Manos Antonakakis,
• Abstract summary: This paper systematizes 84 representative papers for using x86/64 malware sandboxes in the academic literature. We propose a novel framework to simplify sandbox components and organize the literature to derive practical guidelines for using sandboxes.
• Score: 9.24505310582519
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Malware sandboxes provide many benefits for security applications, but they are complex. These complexities can overwhelm new users in different research areas and make it difficult to select, configure, and use sandboxes. Even worse, incorrectly using sandboxes can have a negative impact on security applications. In this paper, we address this knowledge gap by systematizing 84 representative papers for using x86/64 malware sandboxes in the academic literature. We propose a novel framework to simplify sandbox components and organize the literature to derive practical guidelines for using sandboxes. We evaluate the proposed guidelines systematically using three common security applications and demonstrate that the choice of different sandboxes can significantly impact the results. Specifically, our results show that the proposed guidelines improve the sandbox observable activities by at least 1.6x and up to 11.3x. Furthermore, we observe a roughly 25% improvement in accuracy, precision, and recall when using the guidelines to help with a malware family classification task. We conclude by affirming that there is no "silver bullet" sandbox deployment that generalizes, and we recommend that users apply our framework to define a scope for their analysis, a threat model, and derive context about how the sandbox artifacts will influence their intended use case. Finally, it is important that users document their experiment, limitations, and potential solutions for reproducibility

Related papers

• Quantifying Frontier LLM Capabilities for Container Sandbox Escape [1.6245103041408155]
  Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks.<n>To mitigate these risks, agents are commonly deployed and evaluated in isolated "sandbox" environments.<n>We introduce SANDBOXESCAPEBENCH, an open benchmark that safely measures an LLM's capacity to break out of these sandboxes.
  arXiv  Detail & Related papers  (2026-03-01T22:47:39Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation [94.61617176929384]
  OmniSafeBench-MM is a comprehensive toolbox for multi-modal jailbreak attack-defense evaluation.<n>It integrates 13 representative attack methods, 15 defense strategies, and a diverse dataset spanning 9 major risk domains and 50 fine-grained categories.<n>By unifying data, methodology, and evaluation into an open-source, reproducible platform, OmniSafeBench-MM provides a standardized foundation for future research.
  arXiv  Detail & Related papers  (2025-12-06T22:56:29Z)
• SandCell: Sandboxing Rust Beyond Unsafe Code [14.279471205248532]
  Rust is a modern systems programming language that ensures memory safety by enforcing ownership and borrowing rules at compile time.<n>Various approaches for isolating unsafe code to protect safe Rust from vulnerabilities have been proposed.<n>This paper presents SandCell for flexible and lightweight isolation in Rust by leveraging existing syntactic boundaries.
  arXiv  Detail & Related papers  (2025-09-28T19:01:51Z)
• Empirical Security Analysis of Software-based Fault Isolation through Controlled Fault Injection [21.57370108666908]
  Google's V8 heap sandbox protects billions of users across all Chromium-based browsers and countless applications built on Node$.$js and Electron.<n>Despite their widespread use, such SFI mechanisms have seen little security testing.<n>We propose a new testing technique that models the security boundary of modern SFI implementations.
  arXiv  Detail & Related papers  (2025-09-09T13:56:17Z)
• Evaluating Language Model Reasoning about Confidential Information [95.64687778185703]
  We study whether language models exhibit contextual robustness, or the capability to adhere to context-dependent safety specifications.<n>We develop a benchmark (PasswordEval) that measures whether language models can correctly determine when a user request is authorized.<n>We find that current open- and closed-source models struggle with this seemingly simple task, and that, perhaps surprisingly, reasoning capabilities do not generally improve performance.
  arXiv  Detail & Related papers  (2025-08-27T15:39:46Z)
• Threadbox: Sandboxing for Modular Security [0.8594140167290099]
  Threadbox is a sandboxing mechanism that enables having modular and independent sandboxes.<n>We present case studies to illustrate the applicability of the idea and discuss its limitations.
  arXiv  Detail & Related papers  (2025-06-30T10:04:38Z)
• OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents [60.78202583483591]
  We introduce OS-Harm, a new benchmark for measuring safety of computer use agents.<n> OS-Harm is built on top of the OSWorld environment and aims to test models across three categories of harm: deliberate user misuse, prompt injection attacks, and model misbehavior.<n>We evaluate computer use agents based on a range of frontier models and provide insights into their safety.
  arXiv  Detail & Related papers  (2025-06-17T17:59:31Z)
• Playing in the Sandbox: A Study on the Usability of Seccomp [0.8594140167290099]
  We report a usability trial with 7 experienced Seccomp developers exploring how they approached sandboxing an application.<n>We highlight many challenges of using Seccomp, the sandboxing designs by the participants, and what developers think would make it easier for them to sandbox applications effectively.
  arXiv  Detail & Related papers  (2025-06-11T23:27:16Z)
• Unveiling Malware Patterns: A Self-analysis Perspective [15.517313565392852]
  VisUnpack is a static analysis-based data visualization framework for bolstering attack prevention and aiding recovery post-attack. Our method includes unpacking packed malware programs, calculating local similarity descriptors based on basic blocks, enhancing correlations between descriptors, and refining them by minimizing noises. Our comprehensive evaluation of VisUnpack based on a freshly gathered dataset with over 27,106 samples confirms its capability in accurately classifying malware programs with a precision of 99.7%.
  arXiv  Detail & Related papers  (2025-01-10T16:04:13Z)
• Safe Unlearning: A Surprisingly Effective and Generalizable Solution to Defend Against Jailbreak Attacks [89.54736699767315]
  We conjecture that directly unlearn the harmful knowledge in the LLM can be a more effective way to defend against jailbreak attacks. Our solution reduced the Attack Success Rate (ASR) in Vicuna-7B from 82.6% to 7.7% on out-of-distribution (OOD) harmful questions. This significantly outperforms Llama2-7B-Chat, which is fine-tuned on about 0.1M safety alignment samples but still has an ASR of 21.9% even under the help of an additional safety system prompt.
  arXiv  Detail & Related papers  (2024-07-03T07:14:05Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Behaviors [64.9938658716425]
  Existing evaluations of large language models' (LLMs) ability to recognize and reject unsafe user requests face three limitations. First, existing methods often use coarse-grained of unsafe topics, and are over-representing some fine-grained topics. Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations. Third, existing evaluations rely on large LLMs for evaluation, which can be expensive.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)
• Sandboxing Adoption in Open Source Ecosystems [0.8594140167290099]
  This study looks at the use of sandboxing mechanisms in four open-source operating systems. It reveals interesting usage patterns, such as cases where developers simplify their sandbox implementation. It also highlights challenges that may be hindering the widespread adoption of sandboxing mechanisms.
  arXiv  Detail & Related papers  (2024-05-10T12:52:46Z)
• Demystifying Behavior-Based Malware Detection at Endpoints [22.3867935906334]
  We present the first measurement study of the performance of ML-based malware detectors at real-world endpoints. We identify a wide gap between prior methods' sandbox-based detection performance. We propose that yield a relative improvement of 5-30% over the baselines.
  arXiv  Detail & Related papers  (2024-05-09T22:04:55Z)
• On Prompt-Driven Safeguarding for Large Language Models [172.13943777203377]
  We find that in the representation space, the input queries are typically moved by safety prompts in a "higher-refusal" direction. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO. Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries' representations along or opposite the refusal direction, depending on their harmfulness.
  arXiv  Detail & Related papers  (2024-01-31T17:28:24Z)
• Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models [79.0183835295533]
  We introduce the first benchmark for indirect prompt injection attacks, named BIPIA, to assess the risk of such vulnerabilities. Our analysis identifies two key factors contributing to their success: LLMs' inability to distinguish between informational context and actionable instructions, and their lack of awareness in avoiding the execution of instructions within external content. We propose two novel defense mechanisms-boundary awareness and explicit reminder-to address these vulnerabilities in both black-box and white-box settings.
  arXiv  Detail & Related papers  (2023-12-21T01:08:39Z)
• Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks [73.53327403684676]
  We propose an attack-and-defense framework for studying the task of deleting sensitive information directly from model weights. We study direct edits to model weights because this approach should guarantee that particular deleted information is never extracted by future prompt attacks. We show that even state-of-the-art model editing methods such as ROME struggle to truly delete factual information from models like GPT-J, as our whitebox and blackbox attacks can recover "deleted" information from an edited model 38% of the time.
  arXiv  Detail & Related papers  (2023-09-29T17:12:43Z)
• FLIRT: Feedback Loop In-context Red Teaming [79.63896510559357]
  We propose an automatic red teaming framework that evaluates a given black-box model and exposes its vulnerabilities. Our framework uses in-context learning in a feedback loop to red team models and trigger them into unsafe content generation.
  arXiv  Detail & Related papers  (2023-08-08T14:03:08Z)
• XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models [34.75181539924584]
  We introduce a new test suite called XSTest to identify such eXaggerated Safety behaviours. We describe XSTest's creation and composition, and then use the test suite to highlight systematic failure modes in state-of-the-art language models.
  arXiv  Detail & Related papers  (2023-08-02T16:30:40Z)
• Using Static and Dynamic Malware features to perform Malware Ascription [0.0]
  We employ various Static and Dynamic features of malicious executables to classify malware based on their family. We leverage Cuckoo Sandbox and machine learning to make progress in this research.
  arXiv  Detail & Related papers  (2021-12-05T18:01:09Z)
• Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection [67.53296659361598]
  adversarial EXEmples can bypass machine learning-based detection by perturbing relatively few input bytes. We develop a unifying framework that does not only encompass and generalize previous attacks against machine-learning models, but also includes three novel attacks. These attacks, named Full DOS, Extend and Shift, inject the adversarial payload by respectively manipulating the DOS header, extending it, and shifting the content of the first section.
  arXiv  Detail & Related papers  (2020-08-17T07:16:57Z)
• Machine Learning for Offensive Security: Sandbox Classification Using Decision Trees and Artificial Neural Networks [1.758684872705242]
  Machine learning techniques are not reserved for organizations with deep pockets and massive data repositories. This paper aims to give unique insight into how a real offensive team is using machine learning to support offensive operations.
  arXiv  Detail & Related papers  (2020-07-14T01:45:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.