Robustness of LLM-enabled vehicle trajectory prediction under data security threats

• URL: http://arxiv.org/abs/2511.13753v1
• Date: Fri, 14 Nov 2025 03:26:51 GMT
• Title: Robustness of LLM-enabled vehicle trajectory prediction under data security threats
• Authors: Feilong Wang, Fuqiang Liu,
• Abstract summary: Large language models (LLMs) can accurately predict vehicle trajectories and lane-change intentions by gathering and transforming data from surrounding vehicles.<n>This study addresses this gap by conducting a systematic vulnerability analysis of LLM-enabled vehicle trajectory prediction.<n>Experiments on the highD dataset reveal that even minor, physically plausible perturbations can significantly disrupt model outputs.
• Score: 6.812902306426757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of large language models (LLMs) into automated driving systems has opened new possibilities for reasoning and decision-making by transforming complex driving contexts into language-understandable representations. Recent studies demonstrate that fine-tuned LLMs can accurately predict vehicle trajectories and lane-change intentions by gathering and transforming data from surrounding vehicles. However, the robustness of such LLM-based prediction models for safety-critical driving systems remains unexplored, despite the increasing concerns about the trustworthiness of LLMs. This study addresses this gap by conducting a systematic vulnerability analysis of LLM-enabled vehicle trajectory prediction. We propose a one-feature differential evolution attack that perturbs a single kinematic feature of surrounding vehicles within the LLM's input prompts under a black-box setting. Experiments on the highD dataset reveal that even minor, physically plausible perturbations can significantly disrupt model outputs, underscoring the susceptibility of LLM-based predictors to adversarial manipulation. Further analyses reveal a trade-off between accuracy and robustness, examine the failure mechanism, and explore potential mitigation solutions. The findings provide the very first insights into adversarial vulnerabilities of LLM-driven automated vehicle models in the context of vehicular interactions and highlight the need for robustness-oriented design in future LLM-based intelligent transportation systems.

Related papers

• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey [26.44322475984292]
  Trayjectory prediction serves as a critical functionality in autonomous driving.<n>The rise of Large Foundation Models (LFMs) is transforming the research paradigm of trajectory prediction.<n>This article highlights three core methodologies: trajectory-language mapping, multimodal fusion, and constraint-based reasoning.
  arXiv  Detail & Related papers  (2025-09-11T10:30:06Z)
• Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap [51.198001060683296]
  Large Language Models (LLMs) offer transformative potential to address transportation challenges.<n>This survey first presents LLM4TR, a novel conceptual framework that systematically categorizes the roles of LLMs in transportation.<n>For each role, our review spans diverse applications, from traffic prediction and autonomous driving to safety analytics and urban mobility optimization.
  arXiv  Detail & Related papers  (2025-03-27T11:56:27Z)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  We propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge.<n>To explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic.<n>Our Multimodal Retrieval-Augmented Generation model leverages video, control signals, and environmental attributes to learn from past driving experiences.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• Detecting LLM Hallucination Through Layer-wise Information Deficiency: Analysis of Ambiguous Prompts and Unanswerable Questions [60.31496362993982]
  Large language models (LLMs) frequently generate confident yet inaccurate responses.<n>We present a novel, test-time approach to detecting model hallucination through systematic analysis of information flow.
  arXiv  Detail & Related papers  (2024-12-13T16:14:49Z)
• Strada-LLM: Graph LLM for traffic prediction [62.2015839597764]
  A considerable challenge in traffic prediction lies in handling the diverse data distributions caused by vastly different traffic conditions.<n>We propose a graph-aware LLM for traffic prediction that considers proximal traffic information.<n>We adopt a lightweight approach for efficient domain adaptation when facing new data distributions in few-shot fashion.
  arXiv  Detail & Related papers  (2024-10-28T09:19:29Z)
• The Misclassification Likelihood Matrix: Some Classes Are More Likely To Be Misclassified Than Others [1.654278807602897]
  This study introduces Misclassification Likelihood Matrix (MLM) as a novel tool for quantifying the reliability of neural network predictions under distribution shifts. The implications of this work extend beyond image classification, with ongoing applications in autonomous systems, such as self-driving cars.
  arXiv  Detail & Related papers  (2024-07-10T16:43:14Z)
• Using Multimodal Large Language Models for Automated Detection of Traffic Safety Critical Events [5.233512464561313]
  Multimodal Large Language Models (MLLMs) offer a novel approach by integrating textual, visual, and audio modalities. Our framework leverages the reasoning power of MLLMs, directing their output through context-specific prompts. Preliminary results demonstrate the framework's potential in zero-shot learning and accurate scenario analysis.
  arXiv  Detail & Related papers  (2024-06-19T23:50:41Z)
• A Superalignment Framework in Autonomous Driving with Large Language Models [2.650382010271]
  Large language models (LLMs) and multi-modal large language models (MLLMs) are extensively used in autonomous driving. Despite their importance, the security aspect of LLMs in autonomous driving remains underexplored. This research introduces a novel security framework for autonomous vehicles, utilizing a multi-agent LLM approach.
  arXiv  Detail & Related papers  (2024-06-09T05:26:38Z)
• Probing Multimodal LLMs as World Models for Driving [72.18727651074563]
  We look at the application of Multimodal Large Language Models (MLLMs) in autonomous driving. Despite advances in models like GPT-4o, their performance in complex driving environments remains largely unexplored.
  arXiv  Detail & Related papers  (2024-05-09T17:52:42Z)
• LC-LLM: Explainable Lane-Change Intention and Trajectory Predictions with Large Language Models [8.624969693477448]
  Existing motion prediction approaches have ample room for improvement, particularly in terms of long-term prediction accuracy and interpretability. We propose LC-LLM, an explainable lane change prediction model that leverages the strong reasoning capabilities and self-explanation abilities of Large Language Models.
  arXiv  Detail & Related papers  (2024-03-27T08:34:55Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.