Attacking Motion Planners Using Adversarial Perception Errors

• URL: http://arxiv.org/abs/2311.12722v1
• Date: Tue, 21 Nov 2023 16:51:33 GMT
• Title: Attacking Motion Planners Using Adversarial Perception Errors
• Authors: Jonathan Sadeghi, Nicholas A. Lord, John Redford, Romain Mueller
• Abstract summary: We show that it is possible to construct planner inputs that score very highly on various perception quality metrics but still lead to planning failures. We demonstrate the effectiveness of this algorithm by finding attacks for two different black-box planners in several urban and highway driving scenarios.
• Score: 5.423900036420565
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous driving (AD) systems are often built and tested in a modular fashion, where the performance of different modules is measured using task-specific metrics. These metrics should be chosen so as to capture the downstream impact of each module and the performance of the system as a whole. For example, high perception quality should enable prediction and planning to be performed safely. Even though this is true in general, we show here that it is possible to construct planner inputs that score very highly on various perception quality metrics but still lead to planning failures. In an analogy to adversarial attacks on image classifiers, we call such inputs \textbf{adversarial perception errors} and show they can be systematically constructed using a simple boundary-attack algorithm. We demonstrate the effectiveness of this algorithm by finding attacks for two different black-box planners in several urban and highway driving scenarios using the CARLA simulator. Finally, we analyse the properties of these attacks and show that they are isolated in the input space of the planner, and discuss their implications for AD system deployment and testing.

Related papers

• Indiscriminate Disruption of Conditional Inference on Multivariate Gaussians [60.22542847840578]
  Despite advances in adversarial machine learning, inference for Gaussian models in the presence of an adversary is notably understudied. We consider a self-interested attacker who wishes to disrupt a decisionmaker's conditional inference and subsequent actions by corrupting a set of evidentiary variables. To avoid detection, the attacker also desires the attack to appear plausible wherein plausibility is determined by the density of the corrupted evidence.
  arXiv  Detail & Related papers  (2024-11-21T17:46:55Z)
• Self-Supervised Representation Learning for Adversarial Attack Detection [6.528181610035978]
  Supervised learning-based adversarial attack detection methods rely on a large number of labeled data. We propose a self-supervised representation learning framework for the adversarial attack detection task to address this drawback.
  arXiv  Detail & Related papers  (2024-07-05T09:37:16Z)
• DARTH: Holistic Test-time Adaptation for Multiple Object Tracking [87.72019733473562]
  Multiple object tracking (MOT) is a fundamental component of perception systems for autonomous driving. Despite the urge of safety in driving systems, no solution to the MOT adaptation problem to domain shift in test-time conditions has ever been proposed. We introduce DARTH, a holistic test-time adaptation framework for MOT.
  arXiv  Detail & Related papers  (2023-10-03T10:10:42Z)
• EMShepherd: Detecting Adversarial Samples via Side-channel Leakage [6.868995628617191]
  Adversarial attacks have disastrous consequences for deep learning-empowered critical applications. We propose a framework, EMShepherd, to capture electromagnetic traces of model execution, perform processing on traces and exploit them for adversarial detection. We demonstrate that our air-gapped EMShepherd can effectively detect different adversarial attacks on a commonly used FPGA deep learning accelerator.
  arXiv  Detail & Related papers  (2023-03-27T19:38:55Z)
• Physical Passive Patch Adversarial Attacks on Visual Odometry Systems [6.391337032993737]
  We study patch adversarial attacks on visual odometry-based autonomous navigation systems. We show for the first time that the error margin of a visual odometry model can be significantly increased by deploying patch adversarial attacks in the scene.
  arXiv  Detail & Related papers  (2022-07-11T14:41:06Z)
• CARLA-GeAR: a Dataset Generator for a Systematic Evaluation of Adversarial Robustness of Vision Models [61.68061613161187]
  This paper presents CARLA-GeAR, a tool for the automatic generation of synthetic datasets for evaluating the robustness of neural models against physical adversarial patches. The tool is built on the CARLA simulator, using its Python API, and allows the generation of datasets for several vision tasks in the context of autonomous driving. The paper presents an experimental study to evaluate the performance of some defense methods against such attacks, showing how the datasets generated with CARLA-GeAR might be used in future work as a benchmark for adversarial defense in the real world.
  arXiv  Detail & Related papers  (2022-06-09T09:17:38Z)
• Control-Aware Prediction Objectives for Autonomous Driving [78.19515972466063]
  We present control-aware prediction objectives (CAPOs) to evaluate the downstream effect of predictions on control without requiring the planner be differentiable. We propose two types of importance weights that weight the predictive likelihood: one using an attention model between agents, and another based on control variation when exchanging predicted trajectories for ground truth trajectories.
  arXiv  Detail & Related papers  (2022-04-28T07:37:21Z)
• DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows [52.31831255787147]
  We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA) Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution. Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
  arXiv  Detail & Related papers  (2021-05-30T22:07:13Z)
• ExAD: An Ensemble Approach for Explanation-based Adversarial Detection [17.455233006559734]
  We propose ExAD, a framework to detect adversarial examples using an ensemble of explanation techniques. We evaluate our approach using six state-of-the-art adversarial attacks on three image datasets.
  arXiv  Detail & Related papers  (2021-03-22T00:53:07Z)
• Generating and Characterizing Scenarios for Safety Testing of Autonomous Vehicles [86.9067793493874]
  We propose efficient mechanisms to characterize and generate testing scenarios using a state-of-the-art driving simulator. We use our method to characterize real driving data from the Next Generation Simulation (NGSIM) project. We rank the scenarios by defining metrics based on the complexity of avoiding accidents and provide insights into how the AV could have minimized the probability of incurring an accident.
  arXiv  Detail & Related papers  (2021-03-12T17:00:23Z)
• Discovering Avoidable Planner Failures of Autonomous Vehicles using Counterfactual Analysis in Behaviorally Diverse Simulation [16.86782673205523]
  We introduce a planner testing framework that leverages recent progress in simulating behaviorally diverse traffic participants. We show that our method can indeed find a wide range of critical planner failures.
  arXiv  Detail & Related papers  (2020-11-24T09:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.