Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

• URL: http://arxiv.org/abs/2607.28665v1
• Date: Sat, 25 Jul 2026 01:31:37 GMT
• Title: Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems
• Abstract summary: Monitoring systems to independently verify which automated driving system is active are important for safety monitoring, regulatory compliance, insurance assessment, and anomaly detection.<n>We first evaluate the effectiveness of three sequence-based classification models: Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) networks, and a Transformer encoder model.<n>All three models achieve strong clean-data performance with macro F1-scores of 0.92 (GRU), 0.90 (LSTM), and 0.93 (Transformer encoder model) when trained on clean data.
• Score: 0.5156484100374058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automated driving systems (ADSs) are becoming ubiquitous. Future Software Defined Vehicles (SDVs) may be able to run multiple ADSs, both native and aftermarket such as Comma.ai's Openpilot. Monitoring systems to independently verify which automated driving system is active are important for safety monitoring, regulatory compliance, insurance assessment, and anomaly detection. In this paper, we first evaluate the effectiveness of three sequence-based classification models: Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) networks, and a Transformer encoder model for identifying Level 2 automated driving systems using vehicle telematics data alone: Comma Openpilot, Tesla Autopilot, and Cadillac Super Cruise, along with manual driving. All three models achieve strong clean-data performance with macro F1-scores of 0.92 (GRU), 0.90 (LSTM), and 0.93 (Transformer encoder model) when trained on clean data; threat-matched training yields 0.904-0.916 macro F1 with only a modest clean-data penalty. Second, we introduce a modular robustness evaluation framework that simulates realistic telematics degradation through five corruption families at five severity levels (L1-L5). Continuous channels are perturbed using additive white Gaussian noise with cumulative drift, correlated cross-channel noise, and temporal jitter. Binary event signals are subjected to burst loss, delayed transitions, spurious toggles and cross-feature inconsistencies inspired by communication errors. Robustness is measured using macro-F1, which gives equal weight to each class and is suitable for imbalanced multiclass evaluation. Our evaluation reveals a sharp failure-mode split: event-level corruptions reduce macro-F1 only slightly (greater than equal to 0.87 at L5), while temporal jitter collapses macro-F1 to 0.44-0.50 across GRU, LSTM, and Transformer encoder model.

Related papers

• TRACE: A Multi-Layer Benchmark for Human AI Controller Coordination Under Drift and Failure [0.0]
  Modern cyber-physical and AI-assisted systems couple human operators, AI decision modules, and automated controllers in a single control loop.<n>No standard benchmark captures time-aligned, multi-layer traces of how drift and failures propagate across these layers.<n>We construct a benchmark by injecting controlled drift into traces derived from ALFRED, a grounded-instruction benchmark for everyday household tasks.
  arXiv  Detail & Related papers  (2026-08-07T00:03:55Z)
• Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles [0.3324986723090369]
  This paper presents an online anomaly detection framework for autonomous CPS.<n>A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service.<n>A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge.
  arXiv  Detail & Related papers  (2026-07-09T11:48:52Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• Quantum Machine Learning for Cyber-Physical Anomaly Detection in Unmanned Aerial Vehicles: A Leakage-Free Evaluation with Proxy-Audited Feature Sets [0.0]
  Unmanned aerial vehicles (UAVs) are cyber-physical systems whose attack surface spans networked avionics and on-board sensor fusion.<n>We present a leakage-free evaluation of quantum machine learning for UAV anomaly detection on the multi-sensor TLM:UAV benchmark.
  arXiv  Detail & Related papers  (2026-05-19T01:05:16Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Models [50.22099309218635]
  In autonomous driving, closed-loop evaluation has become widely recognized as a more reliable validation method than open-loop evaluation.<n>We present Bench2Drive-VL, an extension of Bench2Drive that brings closed-loop evaluation to VLM-based driving.
  arXiv  Detail & Related papers  (2026-04-01T11:38:46Z)
• AI-Driven Predictive Maintenance with Real-Time Contextual Data Fusion for Connected Vehicles: A Multi-Dataset Evaluation [0.0]
  This paper presents a simulation-validated proof-of-concept framework for V2X-augmented predictive maintenance.<n>It integrates on-board sensor streams with external contextual signals.<n>Edge inference is estimated to reduce latency from 3.5s to under 1.0s versus cloud-only processing.
  arXiv  Detail & Related papers  (2026-03-07T06:08:45Z)
• Benchmarking IoT Time-Series AD with Event-Level Augmentations [34.864214444544565]
  We introduce an evaluation protocol with unified event-level augmentations that simulate real-world issues.<n>We evaluate 14 representative models on five public anomaly datasets.
  arXiv  Detail & Related papers  (2026-02-17T09:45:44Z)
• Multi-modality Anomaly Segmentation on the Road [39.96773593485109]
  Current uni-modal anomaly segmentation frameworks tend to produce high anomaly scores for non-anomalous regions in images.<n>We develop a multi-modal uncertainty-based anomaly segmentation framework, named MMRAS+, for autonomous driving systems.<n>Indeed, MMRAS+ is the first multi-modal anomaly segmentation solution for autonomous driving.
  arXiv  Detail & Related papers  (2025-03-22T09:55:42Z)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  We propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge.<n>To explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic.<n>Our Multimodal Retrieval-Augmented Generation model leverages video, control signals, and environmental attributes to learn from past driving experiences.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• Deep Autoencoders for Unsupervised Anomaly Detection in Wildfire Prediction [42.447827727628734]
  Wildfires pose a significantly increasing hazard to global ecosystems due to the climate crisis. Due to its complex nature, there is an urgent need for innovative approaches to wildfire prediction, such as machine learning. This research took a unique approach, differentiating from classical supervised learning, and addressed the gap in unsupervised wildfire prediction.
  arXiv  Detail & Related papers  (2024-11-14T23:19:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.