Proactive Hardening of LLM Defenses with HASTE

• URL: http://arxiv.org/abs/2601.19051v1
• Date: Tue, 27 Jan 2026 00:19:34 GMT
• Title: Proactive Hardening of LLM Defenses with HASTE
• Authors: Henry Chen, Victor Aranda, Samarth Keshari, Ryan Heartfield, Nicole Nichols,
• Abstract summary: Prompt-based attack techniques are one of the primary challenges in securely deploying and protecting LLM-based AI systems.<n>We present HASTE (Hard-negative Attack Sample Training Engine): a systematic framework that iteratively engineers highly evasive prompts.<n>The framework can be generalized to evaluate prompt-injection detection efficacy, with and without fuzzing, for any hard-negative or hard-positive iteration strategy.
• Score: 0.614338876867286
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Prompt-based attack techniques are one of the primary challenges in securely deploying and protecting LLM-based AI systems. LLM inputs are an unbounded, unstructured space. Consequently, effectively defending against these attacks requires proactive hardening strategies capable of continuously generating adaptive attack vectors to optimize LLM defense at runtime. We present HASTE (Hard-negative Attack Sample Training Engine): a systematic framework that iteratively engineers highly evasive prompts, within a modular optimization process, to continuously enhance detection efficacy for prompt-based attack techniques. The framework is agnostic to synthetic data generation methods, and can be generalized to evaluate prompt-injection detection efficacy, with and without fuzzing, for any hard-negative or hard-positive iteration strategy. Experimental evaluation of HASTE shows that hard negative mining successfully evades baseline detectors, reducing malicious prompt detection for baseline detectors by approximately 64%. However, when integrated with detection model re-training, it optimizes the efficacy of prompt detection models with significantly fewer iteration loops compared to relative baseline strategies. The HASTE framework supports both proactive and reactive hardening of LLM defenses and guardrails. Proactively, developers can leverage HASTE to dynamically stress-test prompt injection detection systems; efficiently identifying weaknesses and strengthening defensive posture. Reactively, HASTE can mimic newly observed attack types and rapidly bridge detection coverage by teaching HASTE-optimized detection models to identify them.

Related papers

• SAGE: Sample-Aware Guarding Engine for Robust Intrusion Detection Against Adversarial Attacks [12.537292017431641]
  Machine learning-based intrusion detection systems (ML-IDS) remain highly susceptible to adversarial attacks.<n>We propose SAGE (Sample-Aware Guarding Engine), a substantially improved defense algorithm that integrates active learning with targeted data reduction.<n> SAGE demonstrates strong predictive performance across multiple intrusion detection datasets, achieving an average F1-score improvement of 201% over the state-of-the-art defenses.
  arXiv  Detail & Related papers  (2025-09-09T18:57:27Z)
• AEGIS : Automated Co-Evolutionary Framework for Guarding Prompt Injections Schema [39.44407870355891]
  We propose AEGIS, an automated co-Evolutionary framework for Guarding prompt Injections.<n>Both attack and defense prompts are iteratively optimized against each other using a gradient-like natural language prompt optimization technique.<n>We evaluate our system on a real-world assignment grading dataset of prompt injection attacks and demonstrate that our method consistently outperforms existing baselines.
  arXiv  Detail & Related papers  (2025-08-27T12:25:45Z)
• A Survey on Model Extraction Attacks and Defenses for Large Language Models [55.60375624503877]
  Model extraction attacks pose significant security threats to deployed language models.<n>This survey provides a comprehensive taxonomy of extraction attacks and defenses, categorizing attacks into functionality extraction, training data extraction, and prompt-targeted attacks.<n>We examine defense mechanisms organized into model protection, data privacy protection, and prompt-targeted strategies, evaluating their effectiveness across different deployment scenarios.
  arXiv  Detail & Related papers  (2025-06-26T22:02:01Z)
• Robust Anti-Backdoor Instruction Tuning in LVLMs [53.766434746801366]
  We introduce a lightweight, certified-agnostic defense framework for large visual language models (LVLMs)<n>Our framework finetunes only adapter modules and text embedding layers under instruction tuning.<n>Experiments against seven attacks on Flickr30k and MSCOCO demonstrate that ours reduces their attack success rate to nearly zero.
  arXiv  Detail & Related papers  (2025-06-04T01:23:35Z)
• RADEP: A Resilient Adaptive Defense Framework Against Model Extraction Attacks [6.6680585862156105]
  We introduce a Resilient Adaptive Defense Framework for Model Extraction Attack Protection (RADEP)<n>RADEP employs progressive adversarial training to enhance model resilience against extraction attempts.<n> Ownership verification is enforced through embedded watermarking and backdoor triggers.
  arXiv  Detail & Related papers  (2025-05-25T23:28:05Z)
• DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks [87.66245688589977]
  LLM-integrated applications and agents are vulnerable to prompt injection attacks.<n>A detection method aims to determine whether a given input is contaminated by an injected prompt.<n>We propose DataSentinel, a game-theoretic method to detect prompt injection attacks.
  arXiv  Detail & Related papers  (2025-04-15T16:26:21Z)
• SHIELD: APT Detection and Intelligent Explanation Using LLM [22.944352324963546]
  Advanced persistent threats (APTs) are sophisticated cyber attacks that can remain undetected for extended periods.<n>Existing provenance-based attack detection methods often lack interpretability and suffer from high false positive rates.<n>We introduce SHIELD, a novel approach that combines statistical anomaly detection and graph-based analysis with the contextual analysis capabilities of large language models.
  arXiv  Detail & Related papers  (2025-02-04T14:20:51Z)
• Attention Tracker: Detecting Prompt Injection Attacks in LLMs [62.247841717696765]
  Large Language Models (LLMs) have revolutionized various domains but remain vulnerable to prompt injection attacks.<n>We introduce the concept of the distraction effect, where specific attention heads shift focus from the original instruction to the injected instruction.<n>We propose Attention Tracker, a training-free detection method that tracks attention patterns on instruction to detect prompt injection attacks.
  arXiv  Detail & Related papers  (2024-11-01T04:05:59Z)
• Defending Large Language Models Against Attacks With Residual Stream Activation Analysis [0.0]
  Large Language Models (LLMs) are vulnerable to adversarial threats.<n>This paper presents an innovative defensive strategy, given white box access to an LLM.<n>We apply a novel methodology for analyzing distinctive activation patterns in the residual streams for attack prompt classification.
  arXiv  Detail & Related papers  (2024-06-05T13:06:33Z)
• Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information [67.78183175605761]
  Large Language Models are susceptible to adversarial prompt attacks. This vulnerability underscores a significant concern regarding the robustness and reliability of LLMs. We introduce a novel approach to detecting adversarial prompts at a token level.
  arXiv  Detail & Related papers  (2023-11-20T03:17:21Z)
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness [53.094682754683255]
  We propose a Model-Agnostic Meta-Attack (MAMA) approach to discover stronger attack algorithms automatically. Our method learns the in adversarial attacks parameterized by a recurrent neural network. We develop a model-agnostic training algorithm to improve the ability of the learned when attacking unseen defenses.
  arXiv  Detail & Related papers  (2021-10-13T13:54:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.