Prompt Injection 2.0: Hybrid AI Threats

• URL: http://arxiv.org/abs/2507.13169v1
• Date: Thu, 17 Jul 2025 14:33:36 GMT
• Title: Prompt Injection 2.0: Hybrid AI Threats
• Authors: Jeremy McHugh, Kristina Šekrst, Jon Cefalu,
• Abstract summary: We build upon Preamble's foundational research and mitigation technologies, evaluating them against contemporary threats.<n>We present architectural solutions that combine prompt isolation, runtime security, and privilege separation with novel threat detection capabilities.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt injection attacks, where malicious input is designed to manipulate AI systems into ignoring their original instructions and following unauthorized commands instead, were first discovered by Preamble, Inc. in May 2022 and responsibly disclosed to OpenAI. Over the last three years, these attacks have continued to pose a critical security threat to LLM-integrated systems. The emergence of agentic AI systems, where LLMs autonomously perform multistep tasks through tools and coordination with other agents, has fundamentally transformed the threat landscape. Modern prompt injection attacks can now combine with traditional cybersecurity exploits to create hybrid threats that systematically evade traditional security controls. This paper presents a comprehensive analysis of Prompt Injection 2.0, examining how prompt injections integrate with Cross-Site Scripting (XSS), Cross-Site Request Forgery (CSRF), and other web security vulnerabilities to bypass traditional security measures. We build upon Preamble's foundational research and mitigation technologies, evaluating them against contemporary threats, including AI worms, multi-agent infections, and hybrid cyber-AI attacks. Our analysis incorporates recent benchmarks that demonstrate how traditional web application firewalls, XSS filters, and CSRF tokens fail against AI-enhanced attacks. We also present architectural solutions that combine prompt isolation, runtime security, and privilege separation with novel threat detection capabilities.

Related papers

• The Hidden Dangers of Browsing AI Agents [0.0]
  This paper presents a comprehensive security evaluation of such agents, focusing on systemic vulnerabilities across multiple architectural layers.<n>Our work outlines the first end-to-end threat model for browsing agents and provides actionable guidance for securing their deployment in real-world environments.
  arXiv  Detail & Related papers  (2025-05-19T13:10:29Z)
• Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents [0.0]
  Decentralized AI agents will soon interact across internet platforms, creating security challenges beyond traditional cybersecurity and AI safety frameworks.<n>We introduce textbfmulti-agent security, a new field dedicated to securing networks of decentralized AI agents against threats that emerge or amplify through their interactions.
  arXiv  Detail & Related papers  (2025-05-04T12:03:29Z)
• AI threats to national security can be countered through an incident regime [55.2480439325792]
  We propose a legally mandated post-deployment AI incident regime that aims to counter potential national security threats from AI systems.<n>Our proposed AI incident regime is split into three phases. The first phase revolves around a novel operationalization of what counts as an 'AI incident'<n>The second and third phases spell out that AI providers should notify a government agency about incidents, and that the government agency should be involved in amending AI providers' security and safety procedures.
  arXiv  Detail & Related papers  (2025-03-25T17:51:50Z)
• A Framework for Evaluating Emerging Cyberattack Capabilities of AI [11.595840449117052]
  This work introduces a novel evaluation framework that addresses limitations by: (1) examining the end-to-end attack chain, (2) identifying gaps in AI threat evaluation, and (3) helping defenders prioritize targeted mitigations.<n>We analyzed over 12,000 real-world instances of AI involvement in cyber incidents, catalogued by Google's Threat Intelligence Group, to curate seven representative attack chain archetypes.<n>We report on AI's potential to amplify offensive capabilities across specific attack stages, and offer recommendations for prioritizing defenses.
  arXiv  Detail & Related papers  (2025-03-14T23:05:02Z)
• CyberSentinel: An Emergent Threat Detection System for AI Security [0.0]
  The rapid advancement of artificial intelligence (AI) has significantly expanded the attack surface for AI-driven cybersecurity threats.<n>This paper introduces CyberSentinel, a unified, single-agent system for emergent threat detection.<n>By continuously adapting to evolving adversarial tactics, CyberSentinel strengthens proactive cybersecurity defense.
  arXiv  Detail & Related papers  (2025-02-20T19:03:32Z)
• Position: Mind the Gap-the Growing Disconnect Between Established Vulnerability Disclosure and AI Security [56.219994752894294]
  We argue that adapting existing processes for AI security reporting is doomed to fail due to fundamental shortcomings for the distinctive characteristics of AI systems.<n>Based on our proposal to address these shortcomings, we discuss an approach to AI security reporting and how the new AI paradigm, AI agents, will further reinforce the need for specialized AI security incident reporting advancements.
  arXiv  Detail & Related papers  (2024-12-19T13:50:26Z)
• Is Generative AI the Next Tactical Cyber Weapon For Threat Actors? Unforeseen Implications of AI Generated Cyber Attacks [0.0]
  This paper delves into the escalating threat posed by the misuse of AI, specifically through the use of Large Language Models (LLMs) Through a series of controlled experiments, the paper demonstrates how these models can be manipulated to bypass ethical and privacy safeguards to effectively generate cyber attacks. We also introduce Occupy AI, a customized, finetuned LLM specifically engineered to automate and execute cyberattacks.
  arXiv  Detail & Related papers  (2024-08-23T02:56:13Z)
• Towards Automated Classification of Attackers' TTPs by combining NLP with ML Techniques [77.34726150561087]
  We evaluate and compare different Natural Language Processing (NLP) and machine learning techniques used for security information extraction in research. Based on our investigations we propose a data processing pipeline that automatically classifies unstructured text according to attackers' tactics and techniques.
  arXiv  Detail & Related papers  (2022-07-18T09:59:21Z)
• A System for Efficiently Hunting for Cyber Threats in Computer Systems Using Threat Intelligence [78.23170229258162]
  We build ThreatRaptor, a system that facilitates cyber threat hunting in computer systems using OSCTI. ThreatRaptor provides (1) an unsupervised, light-weight, and accurate NLP pipeline that extracts structured threat behaviors from unstructured OSCTI text, (2) a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities, and (3) a query synthesis mechanism that automatically synthesizes a TBQL query from the extracted threat behaviors.
  arXiv  Detail & Related papers  (2021-01-17T19:44:09Z)
• Enabling Efficient Cyber Threat Hunting With Cyber Threat Intelligence [94.94833077653998]
  ThreatRaptor is a system that facilitates threat hunting in computer systems using open-source Cyber Threat Intelligence (OSCTI) It extracts structured threat behaviors from unstructured OSCTI text and uses a concise and expressive domain-specific query language, TBQL, to hunt for malicious system activities. Evaluations on a broad set of attack cases demonstrate the accuracy and efficiency of ThreatRaptor in practical threat hunting.
  arXiv  Detail & Related papers  (2020-10-26T14:54:01Z)
• Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain [58.30296637276011]
  This paper summarizes the latest research on adversarial attacks against security solutions based on machine learning techniques. It is the first to discuss the unique challenges of implementing end-to-end adversarial attacks in the cyber security domain.
  arXiv  Detail & Related papers  (2020-07-05T18:22:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.