Hacking Predictors Means Hacking Cars: Using Sensitivity Analysis to Identify Trajectory Prediction Vulnerabilities for Autonomous Driving Security

• URL: http://arxiv.org/abs/2401.10313v2
• Date: Mon, 20 May 2024 23:55:43 GMT
• Title: Hacking Predictors Means Hacking Cars: Using Sensitivity Analysis to Identify Trajectory Prediction Vulnerabilities for Autonomous Driving Security
• Authors: Marsalis Gibson, David Babazadeh, Claire Tomlin, Shankar Sastry,
• Abstract summary: In this paper, we conduct a sensitivity analysis on two trajectory prediction models, Trajectron++ and AgentFormer. The analysis reveals that between all inputs, almost all of the perturbation sensitivities for both models lie only within the most recent position and velocity states. We additionally demonstrate that, despite dominant sensitivity on state history perturbations, an undetectable image map perturbation can induce large prediction error increases in both models.
• Score: 1.949927790632678
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial attacks on learning-based multi-modal trajectory predictors have already been demonstrated. However, there are still open questions about the effects of perturbations on inputs other than state histories, and how these attacks impact downstream planning and control. In this paper, we conduct a sensitivity analysis on two trajectory prediction models, Trajectron++ and AgentFormer. The analysis reveals that between all inputs, almost all of the perturbation sensitivities for both models lie only within the most recent position and velocity states. We additionally demonstrate that, despite dominant sensitivity on state history perturbations, an undetectable image map perturbation made with the Fast Gradient Sign Method can induce large prediction error increases in both models, revealing that these trajectory predictors are, in fact, susceptible to image-based attacks. Using an optimization-based planner and example perturbations crafted from sensitivity results, we show how these attacks can cause a vehicle to come to a sudden stop from moderate driving speeds.

Related papers

• A First Physical-World Trajectory Prediction Attack via LiDAR-induced Deceptions in Autonomous Driving [23.08193005790747]
  Existing attacks compromise the prediction model of a victim AV. We propose a novel two-stage attack framework to realize the single-point attack. Our attack causes a collision rate of up to 63% and various hazardous responses of the victim AV.
  arXiv  Detail & Related papers  (2024-06-17T16:26:00Z)
• Manipulating Trajectory Prediction with Backdoors [94.22382859996453]
  We describe and investigate four triggers that could affect trajectory prediction. The model has good benign performance but is vulnerable to backdoors. We evaluate a range of defenses against backdoors.
  arXiv  Detail & Related papers  (2023-12-21T14:01:51Z)
• Adversarial Attacks Against Uncertainty Quantification [10.655660123083607]
  This work focuses on a different adversarial scenario in which the attacker is still interested in manipulating the uncertainty estimate. In particular, the goal is to undermine the use of machine-learning models when their outputs are consumed by a downstream module or by a human operator.
  arXiv  Detail & Related papers  (2023-09-19T12:54:09Z)
• Adversarial Backdoor Attack by Naturalistic Data Poisoning on Trajectory Prediction in Autonomous Driving [18.72382517467458]
  We propose a novel adversarial backdoor attack against trajectory prediction models. Our attack affects the victim at training time via naturalistic, hence stealthy, poisoned samples crafted using a novel two-step approach. We show that the proposed attack is highly effective, as it can significantly hinder the performance of prediction models.
  arXiv  Detail & Related papers  (2023-06-27T19:15:06Z)
• Robustness Benchmark of Road User Trajectory Prediction Models for Automated Driving [0.0]
  We benchmark machine learning models against perturbations that simulate functional insufficiencies observed during model deployment in a vehicle. Training the models with similar perturbations effectively reduces performance degradation, with error increases of up to +87.5%. We argue that despite being an effective mitigation strategy, data augmentation through perturbations during training does not guarantee robustness towards unforeseen perturbations.
  arXiv  Detail & Related papers  (2023-04-04T15:47:42Z)
• AdvDO: Realistic Adversarial Attacks for Trajectory Prediction [87.96767885419423]
  Trajectory prediction is essential for autonomous vehicles to plan correct and safe driving behaviors. We devise an optimization-based adversarial attack framework to generate realistic adversarial trajectories. Our attack can lead an AV to drive off road or collide into other vehicles in simulation.
  arXiv  Detail & Related papers  (2022-09-19T03:34:59Z)
• Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding [121.66374635092097]
  Trajectories obtained from object detection and tracking are inevitably noisy. We propose a trajectory predictor directly based on detection results without relying on explicitly formed trajectories.
  arXiv  Detail & Related papers  (2022-02-03T09:09:56Z)
• Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks [62.87459235819762]
  In a real-world scenario like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs) This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches.
  arXiv  Detail & Related papers  (2021-08-13T11:49:09Z)
• SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction [64.16212996247943]
  We present a Sparse Graph Convolution Network(SGCN) for pedestrian trajectory prediction. Specifically, the SGCN explicitly models the sparse directed interaction with a sparse directed spatial graph to capture adaptive interaction pedestrians. visualizations indicate that our method can capture adaptive interactions between pedestrians and their effective motion tendencies.
  arXiv  Detail & Related papers  (2021-04-04T03:17:42Z)
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting [91.69900691029908]
  We advocate for predicting both the individual motions as well as the scene occupancy map. We propose a Scene-Actor Graph Neural Network (SA-GNN) which preserves the relative spatial information of pedestrians. On two large-scale real-world datasets, we showcase that our scene-occupancy predictions are more accurate and better calibrated than those from state-of-the-art motion forecasting methods.
  arXiv  Detail & Related papers  (2021-01-07T06:08:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.