From Prompt to Physical Action: Structured Backdoor Attacks on LLM-Mediated Robotic Control Systems

• URL: http://arxiv.org/abs/2604.03890v1
• Date: Sat, 04 Apr 2026 23:06:16 GMT
• Title: From Prompt to Physical Action: Structured Backdoor Attacks on LLM-Mediated Robotic Control Systems
• Abstract summary: Large language models (LLMs) are integrated into robotic control pipelines to translate user prompts into executable commands.<n>This digital-to-physical interface introduces a critical and underexplored vulnerability: structured backdoor attacks embedded during fine-tuning.<n>We experimentally investigate LoRA-based supply-chain backdoors in LLM-mediated ROS2 robotic control systems and evaluate their impact on physical robot execution.
• Score: 3.6185608572339962
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The integration of large language models (LLMs) into robotic control pipelines enables natural language interfaces that translate user prompts into executable commands. However, this digital-to-physical interface introduces a critical and underexplored vulnerability: structured backdoor attacks embedded during fine-tuning. In this work, we experimentally investigate LoRA-based supply-chain backdoors in LLM-mediated ROS2 robotic control systems and evaluate their impact on physical robot execution. We construct two poisoned fine-tuning strategies targeting different stages of the command generation pipeline and reveal a key systems-level insight: back-doors embedded at the natural-language reasoning stage do not reliably propagate to executable control outputs, whereas backdoors aligned directly with structured JSON command formats successfully survive translation and trigger physical actions. In both simulation and real-world experiments, backdoored models achieve an average Attack Success Rate of 83% while maintaining over 93% Clean Performance Accuracy (CPA) and sub-second latency, demonstrating both reliability and stealth. We further implement an agentic verification defense using a secondary LLM for semantic consistency checking. Although this reduces the Attack Success Rate (ASR) to 20%, it increases end-to-end latency to 8-9 seconds, exposing a significant security-responsiveness trade-off in real-time robotic systems. These results highlight structural vulnerabilities in LLM-mediated robotic control architectures and underscore the need for robotics-aware defenses for embodied AI systems.

Related papers

• When Coordination Becomes a Threat: Communication Attacks in LLM-Controlled Multi-Robot Systems [45.96097597345022]
  Large Language Models (LLMs) are increasingly used as high-level planners in embodied multi-robot systems.<n> communication risks in multi-robot collaboration remain insufficiently understood.
  arXiv  Detail & Related papers  (2026-08-07T05:39:27Z)
• Fail-RAG : A Retrieval Augmented Generation Informed Framework for Robot Failure Identification [8.8407924555623]
  Fail-RAG is a Retrieval Augmented Generation (RAG)-based failure detection framework.<n> Fail-RAG achieved 25 percentage point higher failure detection accuracy on average across five types of robot operations.
  arXiv  Detail & Related papers  (2026-06-17T21:02:47Z)
• ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety [54.4092272526747]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success in cross-modal understanding and generation, yet their deployment is threatened by critical safety vulnerabilities.<n>We propose ProjLens, an interpretability framework designed to demystify MLLMs backdoors.
  arXiv  Detail & Related papers  (2026-04-21T04:52:38Z)
• Vision-based Goal-Reaching Control for Mobile Robots Using a Hierarchical Learning Framework [4.014524824655106]
  This paper decomposes the whole system into a set of tightly coupled functional modules.<n>The proposed framework guarantees uniform exponential stability of the actuation system and safety of the whole operation.
  arXiv  Detail & Related papers  (2026-01-02T08:41:47Z)
• Safety Guardrails for LLM-Enabled Robots [82.0459036717193]
  Traditional robot safety approaches do not address the novel vulnerabilities of large language models (LLMs)<n>We propose RoboGuard, a two-stage guardrail architecture to ensure the safety of LLM-enabled robots.<n>We show that RoboGuard reduces the execution of unsafe plans from 92% to below 2.5% without compromising performance on safe plans.
  arXiv  Detail & Related papers  (2025-03-10T22:01:56Z)
• ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models [55.93380086403591]
  Generative large language models are vulnerable to backdoor attacks.<n>$textitELBA-Bench$ allows attackers to inject backdoor through parameter efficient fine-tuning.<n>$textitELBA-Bench$ provides over 1300 experiments.
  arXiv  Detail & Related papers  (2025-02-22T12:55:28Z)
• TrojanRobot: Physical-world Backdoor Attacks Against VLM-based Robotic Manipulation [63.68326640737848]
  textitTrojanRobot is a highly stealthy and broadly effective robotic backdoor attack in the physical world.<n>We introduce a module-poisoning approach by embedding a backdoor module into the modular robotic policy.<n>We develop three types of prime attacks, ie, textitpermutation, textitstagnation, and textitintentional attacks, thus achieving finer-grained backdoors.
  arXiv  Detail & Related papers  (2024-11-18T16:09:26Z)
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics [68.36528819227641]
  This paper systematically evaluates the robustness of Vision-Language-Action (VLA) models.<n>We introduce two untargeted attack objectives that leverage spatial foundations to destabilize robotic actions, and a targeted attack objective that manipulates the robotic trajectory.<n>We design an adversarial patch generation approach that places a small, colorful patch within the camera's view, effectively executing the attack in both digital and physical environments.
  arXiv  Detail & Related papers  (2024-11-18T01:52:20Z)
• Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems [27.316115171846953]
  Large Language Models (LLMs) have shown significant promise in real-world decision-making tasks for embodied AI.<n>LLMs are fine-tuned to leverage their inherent common sense and reasoning abilities while being tailored to specific applications.<n>This fine-tuning process introduces considerable safety and security vulnerabilities, especially in safety-critical cyber-physical systems.
  arXiv  Detail & Related papers  (2024-05-27T17:59:43Z)
• On the Vulnerability of LLM/VLM-Controlled Robotics [54.57914943017522]
  We highlight vulnerabilities in robotic systems integrating large language models (LLMs) and vision-language models (VLMs) due to input modality sensitivities.<n>Our results show that simple input perturbations reduce task execution success rates by 22.2% and 14.6% in two representative LLM/VLM-controlled robotic systems.
  arXiv  Detail & Related papers  (2024-02-15T22:01:45Z)
• Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems [79.07468367923619]
  We propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC) We design an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards. We show that our proposed ActPC agent performs well in the face of sparse (extrinsic) reward signals and is competitive with or outperforms several powerful backprop-based RL approaches.
  arXiv  Detail & Related papers  (2022-09-19T16:49:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.