On Adversarial Robustness of Trajectory Prediction for Autonomous
Vehicles
- URL: http://arxiv.org/abs/2201.05057v1
- Date: Thu, 13 Jan 2022 16:33:04 GMT
- Title: On Adversarial Robustness of Trajectory Prediction for Autonomous
Vehicles
- Authors: Qingzhao Zhang, Shengtuo Hu, Jiachen Sun, Qi Alfred Chen, Z. Morley
Mao
- Abstract summary: Trajectory prediction is a critical component for autonomous vehicles to perform safe planning and navigation.
We propose a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error.
Case studies show that if an adversary drives a vehicle close to the target AV following the adversarial trajectory, the AV may make an inaccurate prediction and make unsafe driving decisions.
- Score: 21.56253104577053
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Trajectory prediction is a critical component for autonomous vehicles (AVs)
to perform safe planning and navigation. However, few studies have analyzed the
adversarial robustness of trajectory prediction or investigated whether the
worst-case prediction can still lead to safe planning. To bridge this gap, we
study the adversarial robustness of trajectory prediction models by proposing a
new adversarial attack that perturbs normal vehicle trajectories to maximize
the prediction error. Our experiments on three models and three datasets show
that the adversarial prediction increases the prediction error by more than
150%. Our case studies show that if an adversary drives a vehicle close to the
target AV following the adversarial trajectory, the AV may make an inaccurate
prediction and even make unsafe driving decisions. We also explore possible
mitigation techniques via data augmentation and trajectory smoothing.
Related papers
- A First Physical-World Trajectory Prediction Attack via LiDAR-induced Deceptions in Autonomous Driving [23.08193005790747]
Existing attacks compromise the prediction model of a victim AV.
We propose a novel two-stage attack framework to realize the single-point attack.
Our attack causes a collision rate of up to 63% and various hazardous responses of the victim AV.
arXiv Detail & Related papers (2024-06-17T16:26:00Z) - Certified Human Trajectory Prediction [66.1736456453465]
Tray prediction plays an essential role in autonomous vehicles.
We propose a certification approach tailored for the task of trajectory prediction.
We address the inherent challenges associated with trajectory prediction, including unbounded outputs, and mutli-modality.
arXiv Detail & Related papers (2024-03-20T17:41:35Z) - Manipulating Trajectory Prediction with Backdoors [94.22382859996453]
We describe and investigate four triggers that could affect trajectory prediction.
The model has good benign performance but is vulnerable to backdoors.
We evaluate a range of defenses against backdoors.
arXiv Detail & Related papers (2023-12-21T14:01:51Z) - Adversarial Backdoor Attack by Naturalistic Data Poisoning on Trajectory
Prediction in Autonomous Driving [18.72382517467458]
We propose a novel adversarial backdoor attack against trajectory prediction models.
Our attack affects the victim at training time via naturalistic, hence stealthy, poisoned samples crafted using a novel two-step approach.
We show that the proposed attack is highly effective, as it can significantly hinder the performance of prediction models.
arXiv Detail & Related papers (2023-06-27T19:15:06Z) - AdvDO: Realistic Adversarial Attacks for Trajectory Prediction [87.96767885419423]
Trajectory prediction is essential for autonomous vehicles to plan correct and safe driving behaviors.
We devise an optimization-based adversarial attack framework to generate realistic adversarial trajectories.
Our attack can lead an AV to drive off road or collide into other vehicles in simulation.
arXiv Detail & Related papers (2022-09-19T03:34:59Z) - Long-Tail Prediction Uncertainty Aware Trajectory Planning for
Self-driving Vehicles [12.645597960926601]
Recent studies have shown that deep learning models trained on a dataset following a long-tailed driving scenario distribution will suffer from large prediction errors in the "tails"
This work defines a notion of prediction model uncertainty to quantify high errors due to sparse data.
Results show that the proposed method can improve the safety of trajectory planning under the prediction uncertainty caused by insufficient data.
arXiv Detail & Related papers (2022-07-02T10:17:31Z) - Uncertainty estimation of pedestrian future trajectory using Bayesian
approximation [137.00426219455116]
Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy.
The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture.
The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
arXiv Detail & Related papers (2022-05-04T04:23:38Z) - Safety-aware Motion Prediction with Unseen Vehicles for Autonomous
Driving [104.32241082170044]
We study a new task, safety-aware motion prediction with unseen vehicles for autonomous driving.
Unlike the existing trajectory prediction task for seen vehicles, we aim at predicting an occupancy map.
Our approach is the first one that can predict the existence of unseen vehicles in most cases.
arXiv Detail & Related papers (2021-09-03T13:33:33Z) - LOKI: Long Term and Key Intentions for Trajectory Prediction [22.097307597204736]
Recent advances in trajectory prediction have shown that explicit reasoning about agents' intent is important to accurately forecast their motion.
We propose LOKI (LOng term and Key Intentions), a novel large-scale dataset that is designed to tackle joint trajectory and intention prediction.
We show our method outperforms state-of-the-art trajectory prediction methods by upto $27%$ and also provide a baseline for frame-wise intention estimation.
arXiv Detail & Related papers (2021-08-18T16:57:03Z) - The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
Roads have well defined geometries, topologies, and traffic rules.
In this paper we propose to incorporate structured priors as a loss function.
We demonstrate the effectiveness of our approach on real-world self-driving datasets.
arXiv Detail & Related papers (2020-06-04T03:56:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.