Efficient and Adaptable Detection of Malicious LLM Prompts via Bootstrap Aggregation

• URL: http://arxiv.org/abs/2602.08062v1
• Date: Sun, 08 Feb 2026 17:11:33 GMT
• Title: Efficient and Adaptable Detection of Malicious LLM Prompts via Bootstrap Aggregation
• Abstract summary: Black-box moderation APIs offer limited transparency and adapt poorly to evolving threats.<n>White-box approaches using large LLM judges impose prohibitive computational costs and require expensive retraining for new attacks.<n>We present BAGEL, a modular, lightweight, and incrementally updatable framework for malicious prompt detection.
• Score: 4.467773944156384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding, reasoning, and generation. However, these systems remain susceptible to malicious prompts that induce unsafe or policy-violating behavior through harmful requests, jailbreak techniques, and prompt injection attacks. Existing defenses face fundamental limitations: black-box moderation APIs offer limited transparency and adapt poorly to evolving threats, while white-box approaches using large LLM judges impose prohibitive computational costs and require expensive retraining for new attacks. Current systems force designers to choose between performance, efficiency, and adaptability. To address these challenges, we present BAGEL (Bootstrap AGgregated Ensemble Layer), a modular, lightweight, and incrementally updatable framework for malicious prompt detection. BAGEL employs a bootstrap aggregation and mixture of expert inspired ensemble of fine-tuned models, each specialized on a different attack dataset. At inference, BAGEL uses a random forest router to identify the most suitable ensemble member, then applies stochastic selection to sample additional members for prediction aggregation. When new attacks emerge, BAGEL updates incrementally by fine-tuning a small prompt-safety classifier (86M parameters) and adding the resulting model to the ensemble. BAGEL achieves an F1 score of 0.92 by selecting just 5 ensemble members (430M parameters), outperforming OpenAI Moderation API and ShieldGemma which require billions of parameters. Performance remains robust after nine incremental updates, and BAGEL provides interpretability through its router's structural features. Our results show ensembles of small finetuned classifiers can match or exceed billion-parameter guardrails while offering the adaptability and efficiency required for production systems.

Related papers

• An Efficient and Modular Framework for Targeted Harm Mitigation in LLMS [3.37470725921294]
  Large Language Models (LLMs) are powerful zero-shot learners but remain prone to misalignment with human preferences.<n>We propose a modular correction framework that augments pretrained LLMs with Activated LoRA adapters and a context-aware routing mechanism.
  arXiv  Detail & Related papers  (2026-09-12T00:24:49Z)
• MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models [53.05463623673949]
  MaskForge is a fully black-box adaptive attack that casts dLLM red-teaming as optimized search over a growing library of structural patterns.<n>It achieves an average attack success rate of 79.3%, a 17.6% improvement over the strongest competing dLLM baseline.
  arXiv  Detail & Related papers  (2026-06-01T18:10:21Z)
• Adversarial Reframing: A Framework for Targeted Generation in Language Models [4.231181719953953]
  Large Language Models (LLMs) are widely deployed in diverse real-world settings, yet remain vulnerable to jailbreaking, where prompt-based attacks safety filters.<n>We present THREAT (edversa bypass generation via Reframing) as a tool for proactively strengthening the safety of foundation models.
  arXiv  Detail & Related papers  (2026-05-20T19:31:07Z)
• PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimization [0.0]
  This paper introduces a novel framework for hardening system prompts through shield appending.<n>We leverage an LLM-as-optimizer to search the space of possible SHIELDs, seeking to minimize a leakage metric derived from a suite of adversarial attacks.<n>We demonstrate empirically that our optimized SHIELDs significantly reduce prompt leakage against a comprehensive set of extraction attacks.
  arXiv  Detail & Related papers  (2025-11-20T10:25:45Z)
• Diffusion LLMs are Natural Adversaries for any LLM [50.88535293540971]
  We introduce a novel framework that transforms the resource-intensive (adversarial) prompt optimization problem into an emphefficient, amortized inference task<n>Our core insight is that pretrained, non-autoregressive generative LLMs, can serve as powerful surrogates for prompt search.<n>We find that the generated prompts are low-perplexity, diverse jailbreaks that exhibit strong transferability to a wide range of black-box target models.
  arXiv  Detail & Related papers  (2025-10-31T19:04:09Z)
• Better Privilege Separation for Agents by Restricting Data Types [6.028799607869068]
  We propose type-directed privilege separation for large language models (LLMs)<n>We restrict the ability of an LLM to interact with third-party data by converting untrusted content to a curated set of data types.<n>Unlike raw strings, each data type is limited in scope and content, eliminating the possibility for prompt injections.
  arXiv  Detail & Related papers  (2025-09-30T08:20:50Z)
• VERA: Variational Inference Framework for Jailbreaking Large Language Models [29.57412296290215]
  API-only access to state-of-the-art LLMs highlights the need for effective black-box jailbreak methods.<n>We introduce VERA: Variational infErence fRamework for jAilbreaking.
  arXiv  Detail & Related papers  (2025-06-27T22:22:00Z)
• MISLEADER: Defending against Model Extraction with Ensembles of Distilled Models [56.09354775405601]
  Model extraction attacks aim to replicate the functionality of a black-box model through query access.<n>Most existing defenses presume that attacker queries have out-of-distribution (OOD) samples, enabling them to detect and disrupt suspicious inputs.<n>We propose MISLEADER, a novel defense strategy that does not rely on OOD assumptions.
  arXiv  Detail & Related papers  (2025-06-03T01:37:09Z)
• ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models [55.93380086403591]
  Generative large language models are vulnerable to backdoor attacks.<n>$textitELBA-Bench$ allows attackers to inject backdoor through parameter efficient fine-tuning.<n>$textitELBA-Bench$ provides over 1300 experiments.
  arXiv  Detail & Related papers  (2025-02-22T12:55:28Z)
• Achieving Byzantine-Resilient Federated Learning via Layer-Adaptive Sparsified Model Aggregation [7.200910949076064]
  Federated Learning (FL) enables multiple clients to collaboratively train a model without sharing their local data. Yet the FL system is vulnerable to well-designed Byzantine attacks, which aim to disrupt the model training process by uploading malicious model updates. We propose the Layer-Adaptive Sparsified Model Aggregation (LASA) approach, which combines pre-aggregation sparsification with layer-wise adaptive aggregation to improve robustness.
  arXiv  Detail & Related papers  (2024-09-02T19:28:35Z)
• Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning [7.239534396071191]
  CapaBoost is a strategy that enhances model capacity by leveraging low-rank updates through parallel weight modules in target layers. We extensively validate the efficacy of CapaBoost through experiments on diverse downstream tasks, including natural language understanding, question answering, and image classification. Our results demonstrate significant improvements over baselines, without incurring additional computation or storage costs.
  arXiv  Detail & Related papers  (2024-07-01T14:26:48Z)
• Learning diverse attacks on large language models for robust red-teaming and safety tuning [126.32539952157083]
  Red-teaming, or identifying prompts that elicit harmful responses, is a critical step in ensuring the safe deployment of large language models.<n>We show that even with explicit regularization to favor novelty and diversity, existing approaches suffer from mode collapse or fail to generate effective attacks.<n>We propose to use GFlowNet fine-tuning, followed by a secondary smoothing phase, to train the attacker model to generate diverse and effective attack prompts.
  arXiv  Detail & Related papers  (2024-05-28T19:16:17Z)
• RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content [62.685566387625975]
  Current mitigation strategies, while effective, are not resilient under adversarial attacks. This paper introduces Resilient Guardrails for Large Language Models (RigorLLM), a novel framework designed to efficiently moderate harmful and unsafe inputs.
  arXiv  Detail & Related papers  (2024-03-19T07:25:02Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.