Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles

• URL: http://arxiv.org/abs/2607.08373v1
• Date: Thu, 09 Jul 2026 11:48:52 GMT
• Title: Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles
• Abstract summary: This paper presents an online anomaly detection framework for autonomous CPS.<n>A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service.<n>A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge.
• Score: 0.3324986723090369
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Connected vehicles are autonomous cyber-physical systems whose behavior must be continuously monitored during operation to detect deviations from normal operation before they propagate into failures. Such evaluation is challenging because the systems themselves evolve: over-the-air updates, configuration changes, and shifting workloads alter the definition of normal behavior, causing static diagnostic methods to degrade silently over time. Existing approaches typically address either automated model adaptation or operator integration in isolation, rather than as a single coordinated supervisory loop. This paper presents an online anomaly detection framework for autonomous CPS that integrates three coordinated mechanisms. A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service, exploiting inter-service dependencies in the microservice topology. An ensemble of three statistical drift detectors monitors the input distribution and raises an alarm only when all three concur, prioritizing precision over recall. A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge while preserving the system's learned response to prior data distributions. The framework is evaluated on a connected-vehicle testbed running an automated valet parking application across seven backend microservices. The attention-augmented agent achieves an F1 score of 0.69, compared to at most 0.11 for any single detector applied uniformly. Following a real software update that induces measurable concept drift, F1 drops to 0.52; after operator-triggered retraining, performance recovers to 0.65 on the new distribution while remaining at 0.69 on the prior one, demonstrating sustained adaptation without catastrophic forgetting.

Related papers

• Concept drift mitigation through community and spectral graph analysis for the detection of cyberattacks in network traffic [1.3999481573773072]
  We introduce t-robustness, a stability score defined for each feature independently of any detection model.<n>It combines the step-by-step distance between successive statistical states of a feature, and its cumulative divergence from its initial state, so that a slow monotonic drift cannot pass for stability.<n>The evaluation is performed on the UGR16 dataset, across three learning scenarios and a control scenario, as well as without model update.
  arXiv  Detail & Related papers  (2026-09-08T20:51:00Z)
• NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness [76.6324710887112]
  NeoHorse-1 is a family of agent-native models developed to explore this path through agentic post-training.<n>Our system combines a heterogeneous model pool with intelligent routing, recording the predicted capability demand, selected service tier, and subsequent interaction for each user turn.
  arXiv  Detail & Related papers  (2026-09-08T03:14:54Z)
• Autonomous Cyber Defense: Real-Time Attack Detection and Mitigation in Software-Defined Networks Using Machine Learning [78.29352305480285]
  In recent intrusions, the interval between initial access and the first lateral movement has been observed to be as short as 27 seconds.<n>This paper presents a closed-loop framework that detects and blocks attacks in software-defined networks without operator involvement.
  arXiv  Detail & Related papers  (2026-08-22T18:38:39Z)
• Behavioral Grammar: Detecting Adaptive Malware via Tiny Language Model Priors and Second-Order Temporal Analysis [1.560152792366037]
  This paper presents Behavioral Grammar, a detection architecture that treats host runtime behavior as a structured language.<n>Each system event is discretized into an 8-token representation spanning event type, process, argument skeleton, path category, parent process, user, destination, and inter-event timing.<n>We frame these findings within a coevolutionary economics model, arguing that behavior-grammar detection shifts the evasion cost from rule circumvention to distribution matching.
  arXiv  Detail & Related papers  (2026-08-01T16:28:38Z)
• Cybersecurity Detection Classification with Reasoning-enabled Language Models [50.72675435043546]
  A major issue in Security Operations Centers (SOCs) is alert fatigue.<n>Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat.<n>We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections.
  arXiv  Detail & Related papers  (2026-07-30T16:22:13Z)
• Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems [0.5156484100374058]
  Monitoring systems to independently verify which automated driving system is active are important for safety monitoring, regulatory compliance, insurance assessment, and anomaly detection.<n>We first evaluate the effectiveness of three sequence-based classification models: Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) networks, and a Transformer encoder model.<n>All three models achieve strong clean-data performance with macro F1-scores of 0.92 (GRU), 0.90 (LSTM), and 0.93 (Transformer encoder model) when trained on clean data.
  arXiv  Detail & Related papers  (2026-07-25T01:31:37Z)
• Validation-Gated Multi-Agent Governance for Online Adaptation of Thermal-Hydraulic Surrogate Models under Operating-Regime Shift [3.732931548972571]
  This study develops a guarded continual-adaptation framework for experimental thermal-hydraulic loop data.<n>The MA-Full mode, in which the role-separated multi-agent council reviews every evaluated stream step, achieved the lowest mean error, 5.72, and 35.8% exceedance.
  arXiv  Detail & Related papers  (2026-06-02T08:31:48Z)
• EAGLE: Edge-Aware Graph Learning for Proactive Delivery Delay Prediction in Smart Logistics Networks [0.5653106385738822]
  We propose a hybrid deep learning framework for proactive supply chain risk management.<n>The proposed method jointly models temporal order-flow dynamics via a lightweight Transformer patch encoder.<n>The framework exhibits a cross-seed F1 standard deviation of only 0.0089 -- a 3.8 times improvement over the best ablated variant.
  arXiv  Detail & Related papers  (2026-04-06T23:35:15Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• Beyond Reward Suppression: Reshaping Steganographic Communication Protocols in MARL via Dynamic Representational Circuit Breaking [0.0]
  steganographic collusion, where agents develop private protocols to evade monitoring, presents a critical AI safety threat.<n>Existing defenses, limited to behavioral or reward layers, fail to detect coordination in latent communication channels.<n>We introduce the Dynamic Representational Circuit Breaker (DRCB), an architectural defense operating at the optimization substrate.
  arXiv  Detail & Related papers  (2026-03-07T04:14:38Z)
• World Model Failure Classification and Anomaly Detection for Autonomous Inspection [23.48742973289626]
  We propose a hybrid framework that combines supervised failure classification with anomaly detection.<n>Our approach uses a world model backbone with compressed video inputs.<n>Experiments show over 90% accuracy in distinguishing between successes, failures, and OOD cases.
  arXiv  Detail & Related papers  (2026-02-18T04:41:14Z)
• Axle Sensor Fusion for Online Continual Wheel Fault Detection in Wayside Railway Monitoring [1.1199585259018456]
  This work presents a semantic-aware, label-efficient continual learning framework for railway fault diagnostics.<n>The model detects minor imperfections due to flats and polygonization, while adapting to evolving operational conditions.
  arXiv  Detail & Related papers  (2026-02-18T00:14:18Z)
• Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
  We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
  arXiv  Detail & Related papers  (2025-10-06T14:48:39Z)
• Lie Detector: Unified Backdoor Detection via Cross-Examination Framework [68.45399098884364]
  We propose a unified backdoor detection framework in the semi-honest setting.<n>Our method achieves superior detection performance, improving accuracy by 5.4%, 1.6%, and 11.9% over SoTA baselines.<n> Notably, it is the first to effectively detect backdoors in multimodal large language models.
  arXiv  Detail & Related papers  (2025-03-21T06:12:06Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Unsupervised Adaptation from Repeated Traversals for Autonomous Driving [54.59577283226982]
  Self-driving cars must generalize to the end-user's environment to operate reliably. One potential solution is to leverage unlabeled data collected from the end-users' environments. There is no reliable signal in the target domain to supervise the adaptation process. We show that this simple additional assumption is sufficient to obtain a potent signal that allows us to perform iterative self-training of 3D object detectors on the target domain.
  arXiv  Detail & Related papers  (2023-03-27T15:07:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.