Towards Physically Realizable Adversarial Attacks in Embodied Vision Navigation

• URL: http://arxiv.org/abs/2409.10071v5
• Date: Fri, 15 Aug 2025 02:26:44 GMT
• Title: Towards Physically Realizable Adversarial Attacks in Embodied Vision Navigation
• Authors: Meng Chen, Jiawei Tu, Chao Qi, Yonghao Dang, Feng Zhou, Wei Wei, Jianqin Yin,
• Abstract summary: We propose a practical attack method for embodied vision navigation by attaching adversarial patches to objects.<n>Our adversarial patches decrease the navigation success rate by an average of 22.39%, outperforming previous methods in practicality, effectiveness, and naturalness.
• Score: 18.218913010189237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The significant advancements in embodied vision navigation have raised concerns about its susceptibility to adversarial attacks exploiting deep neural networks. Investigating the adversarial robustness of embodied vision navigation is crucial, especially given the threat of 3D physical attacks that could pose risks to human safety. However, existing attack methods for embodied vision navigation often lack physical feasibility due to challenges in transferring digital perturbations into the physical world. Moreover, current physical attacks for object detection struggle to achieve both multi-view effectiveness and visual naturalness in navigation scenarios. To address this, we propose a practical attack method for embodied navigation by attaching adversarial patches to objects, where both opacity and textures are learnable. Specifically, to ensure effectiveness across varying viewpoints, we employ a multi-view optimization strategy based on object-aware sampling, which optimizes the patch's texture based on feedback from the vision-based perception model used in navigation. To make the patch inconspicuous to human observers, we introduce a two-stage opacity optimization mechanism, in which opacity is fine-tuned after texture optimization. Experimental results demonstrate that our adversarial patches decrease the navigation success rate by an average of 22.39%, outperforming previous methods in practicality, effectiveness, and naturalness. Code is available at: https://github.com/chen37058/Physical-Attacks-in-Embodied-Nav

Related papers

• Beyond the Patch: Exploring Vulnerabilities of Visuomotor Policies via Viewpoint-Consistent 3D Adversarial Object [26.15314358613966]
  This work proposes a viewpoint-consistent adversarial texture optimization method for 3D objects through differentiable rendering.<n>As optimization strategies, we employ Expectation over Transformation (EOT) with a Coarse-to-Fine (C2F) curriculum.<n>We further integrate saliency-guided perturbations to redirect policy attention and design a targeted loss that persistently drives robots toward adversarial objects.
  arXiv  Detail & Related papers  (2026-03-05T07:57:47Z)
• 3D Gaussian Splatting Driven Multi-View Robust Physical Adversarial Camouflage Generation [50.03578546845548]
  Physical adversarial attack methods expose the vulnerabilities of deep neural networks and pose a significant threat to safety-critical scenarios such as autonomous driving.<n> Camouflage-based physical attack is a more promising approach compared to the patch-based attack, offering stronger adversarial effectiveness in complex physical environments.<n>We propose a physical attack framework based on 3D Gaussian Splatting (3DGS), named PGA, which provides rapid and precise reconstruction with few images.
  arXiv  Detail & Related papers  (2025-07-02T05:10:16Z)
• NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistants [24.689242976554482]
  Navigating unfamiliar environments presents significant challenges for household robots. Existing reinforcement learning methods cannot be directly transferred to new environments. We try to transfer the logical knowledge and the generalization ability of pre-trained foundation models to zero-shot navigation.
  arXiv  Detail & Related papers  (2025-02-19T17:27:47Z)
• Environmental Matching Attack Against Unmanned Aerial Vehicles Object Detection [37.77615360932841]
  Object detection techniques for Unmanned Aerial Vehicles rely on Deep Neural Networks (DNNs) adversarial patches generated by existing algorithms in the UAV domain pay very little attention to the naturalness of adversarial patches. We propose a new method named Environmental Matching Attack(EMA) to address the issue of optimizing the adversarial patch under the constraints of color.
  arXiv  Detail & Related papers  (2024-05-13T09:56:57Z)
• On Inherent Adversarial Robustness of Active Vision Systems [7.803487547944363]
  We show that two active vision methods - GFNet and FALcon - achieve (2-3) times greater robustness compared to a standard passive convolutional network under state-of-the-art adversarial attacks. More importantly, we provide illustrative and interpretable visualization analysis that demonstrates how performing inference from distinct fixation points makes active vision methods less vulnerable to malicious inputs.
  arXiv  Detail & Related papers  (2024-03-29T22:51:45Z)
• Dual Adversarial Resilience for Collaborating Robust Underwater Image Enhancement and Perception [54.672052775549]
  In this work, we introduce a collaborative adversarial resilience network, dubbed CARNet, for underwater image enhancement and subsequent detection tasks. We propose a synchronized attack training strategy with both visual-driven and perception-driven attacks enabling the network to discern and remove various types of attacks. Experiments demonstrate that the proposed method outputs visually appealing enhancement images and perform averagely 6.71% higher detection mAP than state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-09-03T06:52:05Z)
• Unified Adversarial Patch for Visible-Infrared Cross-modal Attacks in the Physical World [11.24237636482709]
  We design a unified adversarial patch that can perform cross-modal physical attacks, achieving evasion in both modalities simultaneously with a single patch. We propose a novel boundary-limited shape optimization approach that aims to achieve compact and smooth shapes for the adversarial patch. Our method is evaluated against several state-of-the-art object detectors, achieving an Attack Success Rate (ASR) of over 80%.
  arXiv  Detail & Related papers  (2023-07-27T08:14:22Z)
• Towards Effective Adversarial Textured 3D Meshes on Physical Face Recognition [42.60954035488262]
  The goal of this work is to develop a more reliable technique that can carry out an end-to-end evaluation of adversarial robustness for commercial systems. We design adversarial textured 3D meshes (AT3D) with an elaborate topology on a human face, which can be 3D-printed and pasted on the attacker's face to evade the defenses. To deviate from the mesh-based space, we propose to perturb the low-dimensional coefficient space based on 3D Morphable Model.
  arXiv  Detail & Related papers  (2023-03-28T08:42:54Z)
• Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning [80.21709045433096]
  A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample. We use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
  arXiv  Detail & Related papers  (2022-11-04T13:54:02Z)
• Benchmarking Adversarial Patch Against Aerial Detection [11.591143898488312]
  A novel adaptive-patch-based physical attack (AP-PA) framework is proposed. AP-PA generates adversarial patches that are adaptive in both physical dynamics and varying scales. We establish one of the first comprehensive, coherent, and rigorous benchmarks to evaluate the attack efficacy of adversarial patches on aerial detection tasks.
  arXiv  Detail & Related papers  (2022-10-30T07:55:59Z)
• Physical Adversarial Attack meets Computer Vision: A Decade Survey [55.38113802311365]
  This paper presents a comprehensive overview of physical adversarial attacks. We take the first step to systematically evaluate the performance of physical adversarial attacks. Our proposed evaluation metric, hiPAA, comprises six perspectives.
  arXiv  Detail & Related papers  (2022-09-30T01:59:53Z)
• Shadows can be Dangerous: Stealthy and Effective Physical-world Adversarial Attack by Natural Phenomenon [79.33449311057088]
  We study a new type of optical adversarial examples, in which the perturbations are generated by a very common natural phenomenon, shadow. We extensively evaluate the effectiveness of this new attack on both simulated and real-world environments.
  arXiv  Detail & Related papers  (2022-03-08T02:40:18Z)
• Developing Imperceptible Adversarial Patches to Camouflage Military Assets From Computer Vision Enabled Technologies [0.0]
  Convolutional neural networks (CNNs) have demonstrated rapid progress and a high level of success in object detection. Recent evidence has highlighted their vulnerability to adversarial attacks. We present a unique method that produces imperceptible patches capable of camouflaging large military assets.
  arXiv  Detail & Related papers  (2022-02-17T20:31:51Z)
• On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Driving [59.33715889581687]
  The existence of real-world adversarial examples (commonly in the form of patches) poses a serious threat for the use of deep learning models in safety-critical computer vision tasks. This paper presents an evaluation of the robustness of semantic segmentation models when attacked with different types of adversarial patches. A novel loss function is proposed to improve the capabilities of attackers in inducing a misclassification of pixels.
  arXiv  Detail & Related papers  (2022-01-05T22:33:43Z)
• Augmented reality navigation system for visual prosthesis [67.09251544230744]
  We propose an augmented reality navigation system for visual prosthesis that incorporates a software of reactive navigation and path planning. It consists on four steps: locating the subject on a map, planning the subject trajectory, showing it to the subject and re-planning without obstacles. Results show how our augmented navigation system help navigation performance by reducing the time and distance to reach the goals, even significantly reducing the number of obstacles collisions.
  arXiv  Detail & Related papers  (2021-09-30T09:41:40Z)
• Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks [62.87459235819762]
  In a real-world scenario like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs) This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches.
  arXiv  Detail & Related papers  (2021-08-13T11:49:09Z)
• Physical world assistive signals for deep neural network classifiers -- neither defense nor attack [23.138996515998347]
  We introduce the concept of Assistive Signals, which are optimized to improve a model's confidence score regardless if it's under attack or not. Experimental evaluations show that the assistive signals generated by our optimization method increase the accuracy and confidence of deep models. We discuss how we can exploit these insights to re-think, or avoid, some patterns that might contribute to, or degrade, the detectability of objects in the real-world.
  arXiv  Detail & Related papers  (2021-05-03T04:02:48Z)
• Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving [87.3492357041748]
  In this paper, we showcase practical susceptibilities of multi-sensor detection by placing an adversarial object on top of a host vehicle. Our experiments demonstrate that successful attacks are primarily caused by easily corrupted image features. Towards more robust multi-modal perception systems, we show that adversarial training with feature denoising can boost robustness to such attacks significantly.
  arXiv  Detail & Related papers  (2021-01-17T21:15:34Z)
• Temporally-Transferable Perturbations: Efficient, One-Shot Adversarial Attacks for Online Visual Object Trackers [81.90113217334424]
  We propose a framework to generate a single temporally transferable adversarial perturbation from the object template image only. This perturbation can then be added to every search image, which comes at virtually no cost, and still, successfully fool the tracker.
  arXiv  Detail & Related papers  (2020-12-30T15:05:53Z)
• Spatiotemporal Attacks for Embodied Agents [119.43832001301041]
  We take the first step to study adversarial attacks for embodied agents. In particular, we generate adversarial examples, which exploit the interaction history in both the temporal and spatial dimensions. Our perturbations have strong attack and generalization abilities.
  arXiv  Detail & Related papers  (2020-05-19T01:38:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.