Manipulating Trajectory Prediction with Backdoors
- URL: http://arxiv.org/abs/2312.13863v2
- Date: Wed, 3 Jan 2024 15:52:24 GMT
- Title: Manipulating Trajectory Prediction with Backdoors
- Authors: Kaouther Messaoud, Kathrin Grosse, Mickael Chen, Matthieu Cord,
Patrick P\'erez, and Alexandre Alahi
- Abstract summary: We describe and investigate four triggers that could affect trajectory prediction.
The model has good benign performance but is vulnerable to backdoors.
We evaluate a range of defenses against backdoors.
- Score: 94.22382859996453
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous vehicles ought to predict the surrounding agents' trajectories to
allow safe maneuvers in uncertain and complex traffic situations. As companies
increasingly apply trajectory prediction in the real world, security becomes a
relevant concern. In this paper, we focus on backdoors - a security threat
acknowledged in other fields but so far overlooked for trajectory prediction.
To this end, we describe and investigate four triggers that could affect
trajectory prediction. We then show that these triggers (for example, a braking
vehicle), when correlated with a desired output (for example, a curve) during
training, cause the desired output of a state-of-the-art trajectory prediction
model. In other words, the model has good benign performance but is vulnerable
to backdoors. This is the case even if the trigger maneuver is performed by a
non-casual agent behind the target vehicle. As a side-effect, our analysis
reveals interesting limitations within trajectory prediction models. Finally,
we evaluate a range of defenses against backdoors. While some, like simple
offroad checks, do not enable detection for all triggers, clustering is a
promising candidate to support manual inspection to find backdoors.
Related papers
- ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving [46.17683799762322]
Trajectory prediction of agents is crucial for the safety of autonomous vehicles.
Previous approaches usually rely on sufficiently long-observed trajectory to predict the future trajectory of the agents.
We propose a general and plug-and-play instantaneous trajectory prediction approach, called ITPNet.
arXiv Detail & Related papers (2024-12-10T10:09:41Z) - A Multi-Loss Strategy for Vehicle Trajectory Prediction: Combining Off-Road, Diversity, and Directional Consistency Losses [68.68514648185828]
Trajectory prediction is essential for the safety and efficiency of planning in autonomous vehicles.
Current models often fail to fully capture complex traffic rules and the complete range of potential vehicle movements.
This study introduces three novel loss functions: Offroad Loss, Direction Consistency Error, and Diversity Loss.
arXiv Detail & Related papers (2024-11-29T14:47:08Z) - Hacking Predictors Means Hacking Cars: Using Sensitivity Analysis to Identify Trajectory Prediction Vulnerabilities for Autonomous Driving Security [1.949927790632678]
In this paper, we conduct a sensitivity analysis on two trajectory prediction models, Trajectron++ and AgentFormer.
The analysis reveals that between all inputs, almost all of the perturbation sensitivities for both models lie only within the most recent position and velocity states.
We additionally demonstrate that, despite dominant sensitivity on state history perturbations, an undetectable image map perturbation can induce large prediction error increases in both models.
arXiv Detail & Related papers (2024-01-18T18:47:29Z) - Trajectory-Prediction with Vision: A Survey [0.0]
Trajectory prediction is an extremely challenging task which recently gained a lot of attention in the autonomous vehicle research community.
A good prediction model can prevent collisions on the road, and hence the ultimate goal for autonomous vehicles: Collision rate: collisions per Million miles.
We categorize the relevant algorithms into different classes so that researchers can follow through the trends in the trajectory-prediction research field.
arXiv Detail & Related papers (2023-03-15T01:06:54Z) - Untargeted Backdoor Attack against Object Detection [69.63097724439886]
We design a poison-only backdoor attack in an untargeted manner, based on task characteristics.
We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
arXiv Detail & Related papers (2022-11-02T17:05:45Z) - On Adversarial Robustness of Trajectory Prediction for Autonomous
Vehicles [21.56253104577053]
Trajectory prediction is a critical component for autonomous vehicles to perform safe planning and navigation.
We propose a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error.
Case studies show that if an adversary drives a vehicle close to the target AV following the adversarial trajectory, the AV may make an inaccurate prediction and make unsafe driving decisions.
arXiv Detail & Related papers (2022-01-13T16:33:04Z) - You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory
Prediction [52.442129609979794]
Recent deep learning approaches for trajectory prediction show promising performance.
It remains unclear which features such black-box models actually learn to use for making predictions.
This paper proposes a procedure that quantifies the contributions of different cues to model performance.
arXiv Detail & Related papers (2021-10-11T14:24:15Z) - TNT: Target-driveN Trajectory Prediction [76.21200047185494]
We develop a target-driven trajectory prediction framework for moving agents.
We benchmark it on trajectory prediction of vehicles and pedestrians.
We outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.
arXiv Detail & Related papers (2020-08-19T06:52:46Z) - Backdoor Learning: A Survey [75.59571756777342]
Backdoor attack intends to embed hidden backdoor into deep neural networks (DNNs)
Backdoor learning is an emerging and rapidly growing research area.
This paper presents the first comprehensive survey of this realm.
arXiv Detail & Related papers (2020-07-17T04:09:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.