CHAI: Command Hijacking against embodied AI

• URL: http://arxiv.org/abs/2510.00181v1
• Date: Tue, 30 Sep 2025 19:02:57 GMT
• Title: CHAI: Command Hijacking against embodied AI
• Authors: Luis Burbano, Diego Ortiz, Qi Sun, Siwei Yang, Haoqin Tu, Cihang Xie, Yinzhi Cao, Alvaro A Cardenas,
• Abstract summary: CHAI (Command Hijacking against embodied AI) is a new class of prompt-based attacks that exploit the multimodal language interpretation abilities of Large Visual-Language Models (LVLMs)<n> CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts.<n>We evaluate CHAI on four LVLM agents; drone emergency landing, autonomous driving, and aerial object tracking, and on a real robotic vehicle.
• Score: 41.9483699766073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied Artificial Intelligence (AI) promises to handle edge cases in robotic vehicle systems where data is scarce by using common-sense reasoning grounded in perception and action to generalize beyond training distributions and adapt to novel real-world situations. These capabilities, however, also create new security risks. In this paper, we introduce CHAI (Command Hijacking against embodied AI), a new class of prompt-based attacks that exploit the multimodal language interpretation abilities of Large Visual-Language Models (LVLMs). CHAI embeds deceptive natural language instructions, such as misleading signs, in visual input, systematically searches the token space, builds a dictionary of prompts, and guides an attacker model to generate Visual Attack Prompts. We evaluate CHAI on four LVLM agents; drone emergency landing, autonomous driving, and aerial object tracking, and on a real robotic vehicle. Our experiments show that CHAI consistently outperforms state-of-the-art attacks. By exploiting the semantic and multimodal reasoning strengths of next-generation embodied AI systems, CHAI underscores the urgent need for defenses that extend beyond traditional adversarial robustness.

Related papers

• Design and Implementation of a Secure RAG-Enhanced AI Chatbot for Smart Tourism Customer Service: Defending Against Prompt Injection Attacks -- A Case Study of Hsinchu, Taiwan [0.0]
  This paper presents the design and implementation of a secure retrieval-augmented generation (RAG) chatbots for Hsinchu smart tourism services.<n>The system integrates RAG with API function calls, multi-layered linguistic analysis, and guardrails against injections.<n> Evaluations with 674 adversarial prompts and 223 benign queries show over 95% accuracy on benign tasks and substantial detection of injection attacks.
  arXiv  Detail & Related papers  (2025-09-22T11:40:29Z)
• ANNIE: Be Careful of Your Robots [48.89876809734855]
  We present the first systematic study of adversarial safety attacks on embodied AI systems.<n>We show attack success rates exceeding 50% across all safety categories.<n>Results expose a previously underexplored but highly consequential attack surface in embodied AI systems.
  arXiv  Detail & Related papers  (2025-09-03T15:00:28Z)
• BounTCHA: A CAPTCHA Utilizing Boundary Identification in Guided Generative AI-extended Videos [4.873950690073118]
  Bots have increasingly been able to bypass most existing CAPTCHA systems, posing significant security threats to web applications.<n>We design and implement BounTCHA, a CAPTCHA mechanism that leverages human perception of boundaries in video transitions and disruptions.<n>We develop a prototype and conduct experiments to collect data on humans' time biases in boundary identification.
  arXiv  Detail & Related papers  (2025-01-30T18:38:09Z)
• Poison Attacks and Adversarial Prompts Against an Informed University Virtual Assistant [3.0874677990361246]
  Large language models (LLMs) are particularly vulnerable to adversarial attacks.<n>The rapid development pace of AI-based systems is being driven by the potential of Generative AI (GenAI) to assist humans in decision making.<n>A threat actor can use security gaps, poor safeguards, and limited data governance to carry out attacks that grant unauthorized access to the system and its data.
  arXiv  Detail & Related papers  (2024-11-03T05:34:38Z)
• A Novel Approach to Guard from Adversarial Attacks using Stable Diffusion [0.0]
  Our proposal suggests a different approach to the AI Guardian framework. Instead of including adversarial examples in the training process, we propose training the AI system without them. This aims to create a system that is inherently resilient to a wider range of attacks.
  arXiv  Detail & Related papers  (2024-05-03T04:08:15Z)
• VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models [65.23688155159398]
  Autoregressive Visual Language Models (VLMs) showcase impressive few-shot learning capabilities in a multimodal context. Recently, multimodal instruction tuning has been proposed to further enhance instruction-following abilities. Adversaries can implant a backdoor by injecting poisoned samples with triggers embedded in instructions or images. We propose a multimodal instruction backdoor attack, namely VL-Trojan.
  arXiv  Detail & Related papers  (2024-02-21T14:54:30Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• BAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Models [54.19289900203071]
  The rise in popularity of text-to-image generative artificial intelligence has attracted widespread public interest. We demonstrate that this technology can be attacked to generate content that subtly manipulates its users. We propose a Backdoor Attack on text-to-image Generative Models (BAGM) Our attack is the first to target three popular text-to-image generative models across three stages of the generative process.
  arXiv  Detail & Related papers  (2023-07-31T08:34:24Z)
• Automating Privilege Escalation with Deep Reinforcement Learning [71.87228372303453]
  In this work, we exemplify the potential threat of malicious actors using deep reinforcement learning to train automated agents. We present an agent that uses a state-of-the-art reinforcement learning algorithm to perform local privilege escalation. Our agent is usable for generating realistic attack sensor data for training and evaluating intrusion detection systems.
  arXiv  Detail & Related papers  (2021-10-04T12:20:46Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.