Manipulating Trajectory Prediction with Backdoors
- URL: http://arxiv.org/abs/2312.13863v2
- Date: Wed, 3 Jan 2024 15:52:24 GMT
- Title: Manipulating Trajectory Prediction with Backdoors
- Authors: Kaouther Messaoud, Kathrin Grosse, Mickael Chen, Matthieu Cord,
Patrick P\'erez, and Alexandre Alahi
- Abstract summary: We describe and investigate four triggers that could affect trajectory prediction.
The model has good benign performance but is vulnerable to backdoors.
We evaluate a range of defenses against backdoors.
- Score: 94.22382859996453
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Autonomous vehicles ought to predict the surrounding agents' trajectories to
allow safe maneuvers in uncertain and complex traffic situations. As companies
increasingly apply trajectory prediction in the real world, security becomes a
relevant concern. In this paper, we focus on backdoors - a security threat
acknowledged in other fields but so far overlooked for trajectory prediction.
To this end, we describe and investigate four triggers that could affect
trajectory prediction. We then show that these triggers (for example, a braking
vehicle), when correlated with a desired output (for example, a curve) during
training, cause the desired output of a state-of-the-art trajectory prediction
model. In other words, the model has good benign performance but is vulnerable
to backdoors. This is the case even if the trigger maneuver is performed by a
non-casual agent behind the target vehicle. As a side-effect, our analysis
reveals interesting limitations within trajectory prediction models. Finally,
we evaluate a range of defenses against backdoors. While some, like simple
offroad checks, do not enable detection for all triggers, clustering is a
promising candidate to support manual inspection to find backdoors.
Related papers
- Hacking Predictors Means Hacking Cars: Using Sensitivity Analysis to Identify Trajectory Prediction Vulnerabilities for Autonomous Driving Security [1.949927790632678]
In this paper, we conduct a sensitivity analysis on two trajectory prediction models, Trajectron++ and AgentFormer.
The analysis reveals that between all inputs, almost all of the perturbation sensitivities for both models lie only within the most recent position and velocity states.
We additionally demonstrate that, despite dominant sensitivity on state history perturbations, an undetectable image map perturbation can induce large prediction error increases in both models.
arXiv Detail & Related papers (2024-01-18T18:47:29Z) - Implicit Occupancy Flow Fields for Perception and Prediction in
Self-Driving [68.95178518732965]
A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants.
Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene.
This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
arXiv Detail & Related papers (2023-08-02T23:39:24Z) - Trajectory-Prediction with Vision: A Survey [0.0]
Trajectory prediction is an extremely challenging task which recently gained a lot of attention in the autonomous vehicle research community.
A good prediction model can prevent collisions on the road, and hence the ultimate goal for autonomous vehicles: Collision rate: collisions per Million miles.
We categorize the relevant algorithms into different classes so that researchers can follow through the trends in the trajectory-prediction research field.
arXiv Detail & Related papers (2023-03-15T01:06:54Z) - Untargeted Backdoor Attack against Object Detection [69.63097724439886]
We design a poison-only backdoor attack in an untargeted manner, based on task characteristics.
We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
arXiv Detail & Related papers (2022-11-02T17:05:45Z) - On Adversarial Robustness of Trajectory Prediction for Autonomous
Vehicles [21.56253104577053]
Trajectory prediction is a critical component for autonomous vehicles to perform safe planning and navigation.
We propose a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error.
Case studies show that if an adversary drives a vehicle close to the target AV following the adversarial trajectory, the AV may make an inaccurate prediction and make unsafe driving decisions.
arXiv Detail & Related papers (2022-01-13T16:33:04Z) - You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory
Prediction [52.442129609979794]
Recent deep learning approaches for trajectory prediction show promising performance.
It remains unclear which features such black-box models actually learn to use for making predictions.
This paper proposes a procedure that quantifies the contributions of different cues to model performance.
arXiv Detail & Related papers (2021-10-11T14:24:15Z) - The PREVENTION Challenge: How Good Are Humans Predicting Lane Changes? [0.0]
In this paper, human's ability to predict lane changes in highway scenarios is analyzed.
Users had to indicate the moment at which they considered that a lane change maneuver was taking place.
Results retrieved have been carefully analyzed and compared to ground truth labels.
arXiv Detail & Related papers (2020-09-11T10:47:07Z) - TNT: Target-driveN Trajectory Prediction [76.21200047185494]
We develop a target-driven trajectory prediction framework for moving agents.
We benchmark it on trajectory prediction of vehicles and pedestrians.
We outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.
arXiv Detail & Related papers (2020-08-19T06:52:46Z) - Backdoor Learning: A Survey [75.59571756777342]
Backdoor attack intends to embed hidden backdoor into deep neural networks (DNNs)
Backdoor learning is an emerging and rapidly growing research area.
This paper presents the first comprehensive survey of this realm.
arXiv Detail & Related papers (2020-07-17T04:09:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.