Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation

• URL: http://arxiv.org/abs/2607.23962v1
• Date: Mon, 27 Jul 2026 03:20:08 GMT
• Title: Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation
• Abstract summary: We develop the first Vision-Language Model (VLM)-based framework for spoofing detection for autonomous vehicles.<n>Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data.<n>Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks.
• Score: 10.15536470870511
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous vehicles (AVs) depend on Global Navigation Satellite Systems (GNSS) for localization and navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first Vision-Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.

Related papers

• Structured Driving-State Narratives for Small Language Model-Based GNSS Spoofing Detection [8.560939383123658]
  This study develops a small language model (SLM)-based framework for detecting and classifying spoofing attacks.<n>The framework converts independent driving states from and other sensing sources into structured semantic narratives.<n>The framework can detect and classify spoofing attacks in real-time while requiring relatively low computational memory resources.
  arXiv  Detail & Related papers  (2026-08-17T19:59:20Z)
• High-Order Liquid Evidence Encoding for Gradual GNSS Spoofing Detection in Autonomous Driving [6.820140586466226]
  Spoofing attacks can manipulate vehicle position estimates.<n>Existing methods often rely on static vehicle-behavior features.<n>We propose a causal high-order liquid evidence framework for spoofing detection.
  arXiv  Detail & Related papers  (2026-08-12T08:30:17Z)
• Instance-level Visual Active Tracking with Occlusion-Aware Planning [61.982298426203165]
  Visual Active Tracking (VAT) aims to control cameras to follow a target in 3D space.<n>VAT faces two key bottlenecks in real-world deployment: confusion from visually similar distractors and severe failure under occlusions.<n>We propose OA-VAT, a unified pipeline with three complementary modules.
  arXiv  Detail & Related papers  (2026-04-23T09:11:50Z)
• Overtake Detection in Trucks Using CAN Bus Signals: A Comparative Study of Machine Learning Methods [51.28632782308621]
  We focus on overtake detection using Controller Area Network (CAN) bus data collected from five in-service trucks provided by the Volvo Group.<n>We evaluate three common classifiers for vehicle manoeuvre detection, Artificial Neural Networks (ANN), Random Forest (RF), and Support Vector Machines (SVM)<n>Our pertruck analysis also reveals that classification accuracy, especially for overtakes, depends on the amount of training data per vehicle.
  arXiv  Detail & Related papers  (2025-07-01T09:20:41Z)
• Tracking Meets Large Multimodal Models for Driving Scenario Understanding [76.71815464110153]
  Large Multimodal Models (LMMs) have recently gained prominence in autonomous driving research.<n>We propose to integrate tracking information as an additional input to recover 3D spatial and temporal details.<n>We introduce a novel approach for embedding this tracking information into LMMs to enhance their understanding of driving scenarios.
  arXiv  Detail & Related papers  (2025-03-18T17:59:12Z)
• Experimental Validation of Sensor Fusion-based GNSS Spoofing Attack Detection Framework for Autonomous Vehicles [5.624009710240032]
  We present a sensor fusion-based spoofing attack detection framework for Autonomous Vehicles. Experiments are conducted in Tuscaloosa, AL, mimicking urban road structures. Results demonstrate the framework's ability to detect various sophisticated spoofing attacks, even including slow drifting attacks.
  arXiv  Detail & Related papers  (2024-01-02T17:30:46Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Unsupervised Adaptation from Repeated Traversals for Autonomous Driving [54.59577283226982]
  Self-driving cars must generalize to the end-user's environment to operate reliably. One potential solution is to leverage unlabeled data collected from the end-users' environments. There is no reliable signal in the target domain to supervise the adaptation process. We show that this simple additional assumption is sufficient to obtain a potent signal that allows us to perform iterative self-training of 3D object detectors on the target domain.
  arXiv  Detail & Related papers  (2023-03-27T15:07:55Z)
• AdvDO: Realistic Adversarial Attacks for Trajectory Prediction [87.96767885419423]
  Trajectory prediction is essential for autonomous vehicles to plan correct and safe driving behaviors. We devise an optimization-based adversarial attack framework to generate realistic adversarial trajectories. Our attack can lead an AV to drive off road or collide into other vehicles in simulation.
  arXiv  Detail & Related papers  (2022-09-19T03:34:59Z)
• A Sensor Fusion-based GNSS Spoofing Attack Detection Framework for Autonomous Vehicles [4.947150829838588]
  This paper presents a sensor fusion based Global Navigation Satellite System (GNSS) spoofing attack detection framework for autonomous vehicles. Data from multiple low-cost in-vehicle sensors are fused and fed into a recurrent neural network model. We have combined k-Nearest Neighbors (k-NN) and Dynamic Time Warping (DTW) algorithms to detect and classify left and right turns.
  arXiv  Detail & Related papers  (2021-08-19T11:59:51Z)
• A Reinforcement Learning Approach for GNSS Spoofing Attack Detection of Autonomous Vehicles [3.918774449495583]
  This paper develops a deep reinforcement learning (RL)-based turn-by-turn spoofing attack detection using low-cost in-vehicle sensor data. We find that the accuracy of the RL model ranges from 99.99% to 100%, and the recall value is 100%. Overall, the analyses reveal that the RL model is effective in turn-by-turn spoofing attack detection.
  arXiv  Detail & Related papers  (2021-08-19T11:48:27Z)
• Sensor Fusion-based GNSS Spoofing Attack Detection Framework for Autonomous Vehicles [4.947150829838588]
  A sensor fusion-based attack detection framework is presented that consists of three concurrent strategies for an autonomous vehicle. Data from multiple low-cost in-vehicle sensors are fused and fed into a recurrent neural network model. We have combined k-Nearest Neighbors (k-NN) and Dynamic Time Warping (DTW) algorithms to detect turns using data from the steering angle sensor. Our analysis reveals that the sensor fusion-based detection framework successfully detects all three types of spoofing attacks within the required computational latency threshold.
  arXiv  Detail & Related papers  (2021-06-05T23:02:55Z)
• Prediction-Based GNSS Spoofing Attack Detection for Autonomous Vehicles [5.579370215490055]
  We have developed a prediction-based spoofing attack detection strategy using the long short-term memory (LSTM) model. Based on the predicted distance traveled between the current location and the immediate future location, a threshold value is established. Our analysis revealed that the prediction-based spoofed attack detection strategy can successfully detect the attack in real-time.
  arXiv  Detail & Related papers  (2020-10-16T18:26:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.