FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruption

• URL: http://arxiv.org/abs/2604.28157v1
• Date: Thu, 30 Apr 2026 17:43:24 GMT
• Title: FlashRT: Towards Computationally and Memory Efficient Red-Teaming for Prompt Injection and Knowledge Corruption
• Abstract summary: Long-context large language models (LLMs) are used to empower many real-world applications, such as retrieval-augmented generation, autonomous agents, and AI assistants.<n>Security remains a major concern for their widespread deployment, with threats such as prompt injection and knowledge corruption.<n>We propose FlashRT, the first framework to improve the efficiency (in terms of both computation and memory) for optimization-based prompt injection and knowledge corruption attacks.
• Score: 20.80189157709862
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-context large language models (LLMs)-for example, Gemini-3.1-Pro and Qwen-3.5-are widely used to empower many real-world applications, such as retrieval-augmented generation, autonomous agents, and AI assistants. However, security remains a major concern for their widespread deployment, with threats such as prompt injection and knowledge corruption. To quantify the security risks faced by LLMs under these threats, the research community has developed heuristic-based and optimization-based red-teaming methods. Optimization-based methods generally produce stronger attacks than heuristic attacks and thus provide a more rigorous assessment of LLM security risks. However, they are often resource-intensive, requiring significant computation and GPU memory, especially for long context scenarios. The resource-intensive nature poses a major obstacle for the community (especially academic researchers) to systematically evaluate the security risks of long-context LLMs and assess the effectiveness of defense strategies at scale. In this work, we propose FlashRT, the first framework to improve the efficiency (in terms of both computation and memory) for optimization-based prompt injection and knowledge corruption attacks under long-context LLMs. Through extensive evaluations, we find that FlashRT consistently delivers a 2x-7x speedup (e.g., reducing runtime from one hour to less than ten minutes) and a 2x-4x reduction in GPU memory consumption (e.g., reducing from 264.1 GB to 65.7 GB GPU memory for a 32K token context) compared to state-of-the-art baseline nanoGCG. FlashRT can be broadly applied to black-box optimization methods, such as TAP and AutoDAN. We hope FlashRT can serve as a red-teaming tool to enable systematic evaluation of long-context LLM security. The code is available at: https://github.com/Wang-Yanting/FlashRT

Related papers

• Understanding and Mitigating Prompt Leaking Attacks in Real-World LLM-Based Applications [57.33208520608409]
  This paper presents a systematic study of prompt leaking in large language model (LLM)-based applications.<n>We find that over 80% of deployments leak system prompts under realistic adversarial queries.<n>We propose AREA, a practical defense that re-anchors the model's attention using an optimizable soft prompt.
  arXiv  Detail & Related papers  (2026-06-17T04:20:00Z)
• Adversarial Reframing: A Framework for Targeted Generation in Language Models [4.231181719953953]
  Large Language Models (LLMs) are widely deployed in diverse real-world settings, yet remain vulnerable to jailbreaking, where prompt-based attacks safety filters.<n>We present THREAT (edversa bypass generation via Reframing) as a tool for proactively strengthening the safety of foundation models.
  arXiv  Detail & Related papers  (2026-05-20T19:31:07Z)
• Analysis of LLMs Against Prompt Injection and Jailbreak Attacks [7.685814179879813]
  This work evaluates prompt-injection and jailbreak vulnerability using a large, manually curated dataset.<n>We observe significant behavioural variation across models, including refusal responses and complete silent non-responsiveness triggered by internal safety mechanisms.
  arXiv  Detail & Related papers  (2026-02-24T12:32:11Z)
• PSM: Prompt Sensitivity Minimization via LLM-Guided Black-Box Optimization [0.0]
  This paper introduces a novel framework for hardening system prompts through shield appending.<n>We leverage an LLM-as-optimizer to search the space of possible SHIELDs, seeking to minimize a leakage metric derived from a suite of adversarial attacks.<n>We demonstrate empirically that our optimized SHIELDs significantly reduce prompt leakage against a comprehensive set of extraction attacks.
  arXiv  Detail & Related papers  (2025-11-20T10:25:45Z)
• Are My Optimized Prompts Compromised? Exploring Vulnerabilities of LLM-based Optimizers [21.207996237794855]
  We present the first systematic analysis of poisoning risks in LLM-based prompt optimization.<n>We find systems are substantially more vulnerable to manipulated feedback than to injected queries.<n>We propose a lightweight highlighting defense that reduces the fake-reward $Delta$ASR from 0.23 to 0.07 without degrading utility.
  arXiv  Detail & Related papers  (2025-10-16T07:28:54Z)
• Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security [0.0]
  Large language models (LLMs) increasingly integrate native code interpreters, enabling real-time execution capabilities.<n>These integrations introduce potential system-level cybersecurity threats, fundamentally different from prompt-based vulnerabilities.<n>We propose CIRCLE (Code-Interpreter Resilience Check for LLM Exploits), a simple benchmark comprising 1,260 prompts targeting CPU, memory, and disk resource exhaustion.
  arXiv  Detail & Related papers  (2025-07-25T16:06:16Z)
• ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning [64.32925552574115]
  ARMOR is a large language model that analyzes jailbreak strategies and extracts the core intent.<n> ARMOR achieves state-of-the-art safety performance, with an average harmful rate of 0.002 and an attack success rate of 0.06 against advanced optimization-based jailbreaks.
  arXiv  Detail & Related papers  (2025-07-14T09:05:54Z)
• CoP: Agentic Red-teaming for Large Language Models using Composition of Principles [68.73212422583548]
  This paper proposes an agentic workflow to automate and scale the red-teaming process of Large Language Models (LLMs)<n>Human users provide a set of red-teaming principles as instructions to an AI agent to automatically orchestrate effective red-teaming strategies and generate jailbreak prompts.<n>When tested against leading LLMs, CoP reveals unprecedented safety risks by finding novel jailbreak prompts and improving the best-known single-turn attack success rate by up to 19.0 times.
  arXiv  Detail & Related papers  (2025-06-01T02:18:41Z)
• PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization [13.751251342738225]
  Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of applications.<n>They also exhibit inherent limitations, such as outdated knowledge and susceptibility to hallucinations.<n>Recent efforts have focused on the security of RAG-based LLMs, yet existing attack methods face three critical challenges.<n>We propose coordinated Prompt-RAG attack (PR-attack), a novel optimization-driven attack that introduces a small number of poisoned texts into the knowledge database.
  arXiv  Detail & Related papers  (2025-04-10T13:09:50Z)
• LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models [59.29840790102413]
  Existing jailbreak attacks are primarily based on opaque optimization techniques and gradient search methods.<n>We propose LLM-Virus, a jailbreak attack method based on evolutionary algorithm, termed evolutionary jailbreak.<n>Our results show that LLM-Virus achieves competitive or even superior performance compared to existing attack methods.
  arXiv  Detail & Related papers  (2024-12-28T07:48:57Z)
• AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases [73.04652687616286]
  We propose AgentPoison, the first backdoor attack targeting generic and RAG-based LLM agents by poisoning their long-term memory or RAG knowledge base. Unlike conventional backdoor attacks, AgentPoison requires no additional model training or fine-tuning. On each agent, AgentPoison achieves an average attack success rate higher than 80% with minimal impact on benign performance.
  arXiv  Detail & Related papers  (2024-07-17T17:59:47Z)
• ShadowCode: Towards (Automatic) External Prompt Injection Attack against Code LLMs [56.46702494338318]
  This paper introduces a new attack paradigm: (automatic) external prompt injection against code-oriented large language models.<n>We propose ShadowCode, a simple yet effective method that automatically generates induced perturbations based on code simulation.<n>We evaluate our method across 13 distinct malicious objectives, generating 31 threat cases spanning three popular programming languages.
  arXiv  Detail & Related papers  (2024-07-12T10:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.