Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attacks

• URL: http://arxiv.org/abs/2412.02795v1
• Date: Tue, 03 Dec 2024 19:54:32 GMT
• Title: Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attacks
• Authors: Zijiao Yang, Xiangxi Shi, Eric Slyman, Stefan Lee,
• Abstract summary: Vision-and-Language Navigation (VLN) task as a representative setting.<n>Whitebox adversarial attack developed to induce desired behaviors in pretrained VLN agents.<n>We find our attacks can induce early-termination behaviors or divert an agent along an attacker-defined multi-step trajectory.
• Score: 12.96291706848273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Assistive embodied agents that can be instructed in natural language to perform tasks in open-world environments have the potential to significantly impact labor tasks like manufacturing or in-home care -- benefiting the lives of those who come to depend on them. In this work, we consider how this benefit might be hijacked by local modifications in the appearance of the agent's operating environment. Specifically, we take the popular Vision-and-Language Navigation (VLN) task as a representative setting and develop a whitebox adversarial attack that optimizes a 3D attack object's appearance to induce desired behaviors in pretrained VLN agents that observe it in the environment. We demonstrate that the proposed attack can cause VLN agents to ignore their instructions and execute alternative actions after encountering the attack object -- even for instructions and agent paths not considered when optimizing the attack. For these novel settings, we find our attacks can induce early-termination behaviors or divert an agent along an attacker-defined multi-step trajectory. Under both conditions, environmental attacks significantly reduce agent capabilities to successfully follow user instructions.

Related papers

• In-Context Defense in Computer Agents: An Empirical Study [19.734768644310414]
  We introduce textbfin-context defense, leveraging in-context learning and chain-of-thought reasoning to counter such attacks. Our approach involves augmenting the agent's context with a small set of carefully curated exemplars containing both malicious environments and corresponding defensive responses. Experiments demonstrate the effectiveness of our method, reducing attack success rates by 91.2% on pop-up window attacks, 74.6% on average on environment injection attacks, while achieving 100% successful defenses against distracting advertisements.
  arXiv  Detail & Related papers  (2025-03-12T10:38:15Z)
• AIM: Additional Image Guided Generation of Transferable Adversarial Attacks [72.24101555828256]
  Transferable adversarial examples highlight the vulnerability of deep neural networks (DNNs) to imperceptible perturbations across various real-world applications. In this work, we focus on generative approaches for targeted transferable attacks. We introduce a novel plug-and-play module into the general generator architecture to enhance adversarial transferability.
  arXiv  Detail & Related papers  (2025-01-02T07:06:49Z)
• Adversarial Inception for Bounded Backdoor Poisoning in Deep Reinforcement Learning [16.350898218047405]
  We propose a new class of backdoor attacks against Deep Reinforcement Learning (DRL) algorithms. These attacks achieve state of the art performance while minimally altering the agent's rewards. We then devise an online attack which significantly out-performs prior attacks under bounded reward constraints.
  arXiv  Detail & Related papers  (2024-10-17T19:50:28Z)
• Compromising Embodied Agents with Contextual Backdoor Attacks [69.71630408822767]
  Large language models (LLMs) have transformed the development of embodied intelligence. This paper uncovers a significant backdoor security threat within this process. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM.
  arXiv  Detail & Related papers  (2024-08-06T01:20:12Z)
• HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments [93.94020724735199]
  HAZARD consists of three unexpected disaster scenarios, including fire, flood, and wind. This benchmark enables us to evaluate autonomous agents' decision-making capabilities across various pipelines.
  arXiv  Detail & Related papers  (2024-01-23T18:59:43Z)
• Pre-trained Trojan Attacks for Visual Recognition [106.13792185398863]
  Pre-trained vision models (PVMs) have become a dominant component due to their exceptional performance when fine-tuned for downstream tasks. We propose the Pre-trained Trojan attack, which embeds backdoors into a PVM, enabling attacks across various downstream vision tasks. We highlight the challenges posed by cross-task activation and shortcut connections in successful backdoor attacks.
  arXiv  Detail & Related papers  (2023-12-23T05:51:40Z)
• Attack-SAM: Towards Attacking Segment Anything Model With Adversarial Examples [68.5719552703438]
  Segment Anything Model (SAM) has attracted significant attention recently, due to its impressive performance on various downstream tasks. Deep vision models are widely recognized as vulnerable to adversarial examples, which fool the model to make wrong predictions with imperceptible perturbation. This work is the first of its kind to conduct a comprehensive investigation on how to attack SAM with adversarial examples.
  arXiv  Detail & Related papers  (2023-05-01T15:08:17Z)
• Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics [57.671493865825255]
  We propose to model the impact of actions on-the-fly using latent embeddings. By combining these latent action embeddings with a novel, transformer-based, policy head, we design an Action Adaptive Policy. We show that our AAP is highly performant even when faced, at inference-time with missing actions and, previously unseen, perturbed action space.
  arXiv  Detail & Related papers  (2023-04-24T17:35:47Z)
• Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks [76.35478518372692]
  We introduce epsilon-illusory, a novel form of adversarial attack on sequential decision-makers. Compared to existing attacks, we empirically find epsilon-illusory to be significantly harder to detect with automated methods. Our findings suggest the need for better anomaly detectors, as well as effective hardware- and system-level defenses.
  arXiv  Detail & Related papers  (2022-07-20T19:49:09Z)
• Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns [18.694795507945603]
  Recent studies demonstrated the vulnerability of control policies learned through deep reinforcement learning against adversarial attacks. This paper investigates the feasibility of targeted attacks through visually learned patterns placed on physical object in the environment.
  arXiv  Detail & Related papers  (2021-09-16T04:59:06Z)
• Policy Teaching in Reinforcement Learning via Environment Poisoning Attacks [33.41280432984183]
  We study a security threat to reinforcement learning where an attacker poisons the learning environment to force the agent into executing a target policy chosen by the attacker. As a victim, we consider RL agents whose objective is to find a policy that maximizes reward in infinite-horizon problem settings.
  arXiv  Detail & Related papers  (2020-11-21T16:54:45Z)
• Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning [33.41280432984183]
  We study a security threat to reinforcement learning where an attacker poisons the learning environment to force the agent into executing a target policy. As a victim, we consider RL agents whose objective is to find a policy that maximizes average reward in undiscounted infinite-horizon problem settings.
  arXiv  Detail & Related papers  (2020-03-28T23:22:28Z)
• Counterfactual Vision-and-Language Navigation via Adversarial Path Sampling [65.99956848461915]
  Vision-and-Language Navigation (VLN) is a task where agents must decide how to move through a 3D environment to reach a goal. One of the problems of the VLN task is data scarcity since it is difficult to collect enough navigation paths with human-annotated instructions for interactive environments. We propose an adversarial-driven counterfactual reasoning model that can consider effective conditions instead of low-quality augmented data.
  arXiv  Detail & Related papers  (2019-11-17T18:02:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.