Cascade: Composing Software-Hardware Attack Gadgets for Adversarial Threat Amplification in Compound AI Systems

• URL: http://arxiv.org/abs/2603.12023v1
• Date: Thu, 12 Mar 2026 15:03:51 GMT
• Title: Cascade: Composing Software-Hardware Attack Gadgets for Adversarial Threat Amplification in Compound AI Systems
• Abstract summary: Compound AI systems are constructed on a layered traditional software stack running on a distributed hardware infrastructure.<n>This work investigates how traditional software and hardware vulnerabilities can complement algorithmic attacks to compromise the integrity of a compound AI system.
• Score: 2.393556797236307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rapid progress in generative AI has given rise to Compound AI systems - pipelines comprised of multiple large language models (LLM), software tools and database systems. Compound AI systems are constructed on a layered traditional software stack running on a distributed hardware infrastructure. Many of the diverse software components are vulnerable to traditional security flaws documented in the Common Vulnerabilities and Exposures (CVE) database, while the underlying distributed hardware infrastructure remains exposed to timing attacks, bit-flip faults, and power-based side channels. Today, research targets LLM-specific risks like model extraction, training data leakage, and unsafe generation -- overlooking the impact of traditional system vulnerabilities. This work investigates how traditional software and hardware vulnerabilities can complement LLM-specific algorithmic attacks to compromise the integrity of a compound AI pipeline. We demonstrate two novel attacks that combine system-level vulnerabilities with algorithmic weaknesses: (1) Exploiting a software code injection flaw along with a guardrail Rowhammer attack to inject an unaltered jailbreak prompt into an LLM, resulting in an AI safety violation, and (2) Manipulating a knowledge database to redirect an LLM agent to transmit sensitive user data to a malicious application, thus breaching confidentiality. These attacks highlight the need to address traditional vulnerabilities; we systematize the attack primitives and analyze their composition by grouping vulnerabilities by their objective and mapping them to distinct stages of an attack lifecycle. This approach enables a rigorous red-teaming exercise and lays the groundwork for future defense strategies.

Related papers

• Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw [42.9467373594646]
  This report presents a comprehensive security analysis of the OpenClaw ecosystem.<n>We highlight critical vulnerabilities such as prompt injection-driven Remote Code Execution (RCE), sequential tool attack chains, context amnesia, and supply chain contamination.<n>To address these systemic architectural flaws, we introduce the Full-Lifecycle Agent Security Architecture (FASA)<n>This theoretical defense blueprint advocates for zero-trust agentic execution, dynamic intent verification, and cross-layer reasoning-action correlation.
  arXiv  Detail & Related papers  (2026-03-13T04:33:05Z)
• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multi-Step Malware [6.5249834967502744]
  Large language model (LLM)-based systems have created a new attack surface that existing security frameworks inadequately address.<n>We propose that attacks targeting LLM-based applications constitute a distinct class of malware, which we term textitpromptware, and introduce a five-step kill chain model for analyzing these threats.
  arXiv  Detail & Related papers  (2026-01-14T16:57:04Z)
• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search [58.8834056209347]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs.<n>We introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base.
  arXiv  Detail & Related papers  (2025-12-01T07:05:23Z)
• Secure Tug-of-War (SecTOW): Iterative Defense-Attack Training with Reinforcement Learning for Multimodal Model Security [63.41350337821108]
  We propose Secure Tug-of-War (SecTOW) to enhance the security of multimodal large language models (MLLMs)<n>SecTOW consists of two modules: a defender and an auxiliary attacker, both trained iteratively using reinforcement learning (GRPO)<n>We show that SecTOW significantly improves security while preserving general performance.
  arXiv  Detail & Related papers  (2025-07-29T17:39:48Z)
• Prompt Injection 2.0: Hybrid AI Threats [0.0]
  We build upon Preamble's foundational research and mitigation technologies, evaluating them against contemporary threats.<n>We present architectural solutions that combine prompt isolation, runtime security, and privilege separation with novel threat detection capabilities.
  arXiv  Detail & Related papers  (2025-07-17T14:33:36Z)
• ATAG: AI-Agent Application Threat Assessment with Attack Graphs [23.757154032523093]
  This paper introduces AI-agent application Threat assessment with Attack Graphs (ATAG)<n>ATAG is a novel framework designed to systematically analyze the security risks associated with AI-agent applications.<n>It facilitates proactive identification and mitigation of AI-agent threats in multi-agent applications.
  arXiv  Detail & Related papers  (2025-06-03T13:25:40Z)
• CRAFT: Characterizing and Root-Causing Fault Injection Threats at Pre-Silicon [3.6158033114580674]
  Fault injection attacks pose significant security threats to embedded systems.<n>Early detection and understanding of how physical faults propagate to system-level behavior are essential to safeguarding cyberinfrastructure.<n>This work introduces CRAFT, a framework that combines pre-silicon analysis with post-silicon validation to systematically uncover and analyze fault injection vulnerabilities.
  arXiv  Detail & Related papers  (2025-03-05T20:17:46Z)
• Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks [88.84977282952602]
  A high volume of recent ML security literature focuses on attacks against aligned large language models (LLMs)<n>In this paper, we analyze security and privacy vulnerabilities that are unique to LLM agents.<n>We conduct a series of illustrative attacks on popular open-source and commercial agents, demonstrating the immediate practical implications of their vulnerabilities.
  arXiv  Detail & Related papers  (2025-02-12T17:19:36Z)
• POEX: Towards Policy Executable Jailbreak Attacks Against the LLM-based Robots [12.356183647226306]
  In this paper, we investigate the feasibility and rationale of jailbreak attacks against LLM-based robots.<n>To enable a more thorough security analysis, we introduce POEX (POlicy EXecutable) jailbreak.<n> Experiments conducted on the real-world robotic systems and in simulation demonstrate the efficacy of POEX.
  arXiv  Detail & Related papers  (2024-12-21T13:58:27Z)
• SoK: A Systems Perspective on Compound AI Threats and Countermeasures [3.458371054070399]
  We discuss different software and hardware attacks applicable to compound AI systems. We show how combining multiple attack mechanisms can reduce the threat model assumptions required for an isolated attack.
  arXiv  Detail & Related papers  (2024-11-20T17:08:38Z)
• h4rm3l: A language for Composable Jailbreak Attack Synthesis [48.5611060845958]
  h4rm3l is a novel approach that addresses the gap with a human-readable domain-specific language.<n>We show that h4rm3l's synthesized attacks are diverse and more successful than existing jailbreak attacks in literature.
  arXiv  Detail & Related papers  (2024-08-09T01:45:39Z)
• Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection [64.67495502772866]
  Large Language Models (LLMs) are increasingly being integrated into various applications. We show how attackers can override original instructions and employed controls using Prompt Injection attacks. We derive a comprehensive taxonomy from a computer security perspective to systematically investigate impacts and vulnerabilities.
  arXiv  Detail & Related papers  (2023-02-23T17:14:38Z)
• Towards Automated Classification of Attackers' TTPs by combining NLP with ML Techniques [77.34726150561087]
  We evaluate and compare different Natural Language Processing (NLP) and machine learning techniques used for security information extraction in research. Based on our investigations we propose a data processing pipeline that automatically classifies unstructured text according to attackers' tactics and techniques.
  arXiv  Detail & Related papers  (2022-07-18T09:59:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.