CAN-QA: A Question-Answering Benchmark for Reasoning over In-Vehicle CAN Traffic

• URL: http://arxiv.org/abs/2604.24935v1
• Date: Mon, 27 Apr 2026 19:20:59 GMT
• Title: CAN-QA: A Question-Answering Benchmark for Reasoning over In-Vehicle CAN Traffic
• Abstract summary: We introduce CAN-QA, the first benchmark that reformulates CAN traffic analysis as a question-answering (QA) task.<n>Can-QA converts raw CAN logs into temporally segmented windows and applies deterministic rule-based templates to generate natural-language questions.<n>Our results indicate that, although these models capture superficial statistical regularities, they struggle with temporal reasoning, multi-condition inference, and higher-level behavioral interpretation.
• Score: 12.047310162349584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Controller Area Network (CAN) is a safety-critical in-vehicle communication protocol that lacks built-in security mechanisms, making intrusion detection essential. Existing approaches predominantly formulate CAN intrusion detection as a classification task, mapping complex traffic patterns to attack labels. However, this formulation abstracts away the temporal and relational structure of CAN traffic and misaligns with real-world forensic workflows, which require systematic reasoning about traffic behavior. To address this gap, we introduce CAN-QA, the first benchmark that reformulates CAN traffic analysis as a question-answering (QA) task. CAN-QA converts raw CAN logs into temporally segmented windows and applies deterministic rule-based templates to generate natural-language questions paired with automatically derived ground-truth answers. The resulting dataset comprises 33,128 QA pairs across 10 categories, each targeting distinct semantic and temporal properties of CAN traffic. Using CAN-QA, we evaluate large language models across both True/False and multiple-choice formats. Our results indicate that, although these models capture superficial statistical regularities, they struggle with temporal reasoning, multi-condition inference, and higher-level behavioral interpretation. Our code is available at https://github.com/Kriiiiss/CAN-QA.

Related papers

• Autonomous Cyber Defense: Real-Time Attack Detection and Mitigation in Software-Defined Networks Using Machine Learning [78.29352305480285]
  In recent intrusions, the interval between initial access and the first lateral movement has been observed to be as short as 27 seconds.<n>This paper presents a closed-loop framework that detects and blocks attacks in software-defined networks without operator involvement.
  arXiv  Detail & Related papers  (2026-08-22T18:38:39Z)
• Stateful Multi-Agent LLMs for Cross-View Interface Alignment in Automotive Model-Based Systems Engineering [29.585470064353014]
  Large Language Models (LLMs) can accelerate Model-Based Systems Engineering (MBSE) for software-defined vehicles.<n>Their probabilistic nature causes "architectural drift", fabricating interfaces in behavioral views that lack structural foundations.<n>We propose a stateful, multi-agent validation pipeline to enforce deterministic interface alignment.
  arXiv  Detail & Related papers  (2026-08-08T09:49:10Z)
• Conflict-Free Policy Languages for Probabilistic ML Predicates: A Framework and Case Study with the Semantic Router DSL [11.868735376461446]
  A class of routing and access-control systems base their decisions on probabilistic ML signals.<n>Two such signals, declared over categories the author intended to be disjoint, can both clear their thresholds on the same query and silently route it to the wrong model.<n>We show that for the embedding case, which dominates in practice, replacing independent thresholding with a temperature-scaled softmax partitions the embedding space into Voronoi regions where co-firing is impossible.
  arXiv  Detail & Related papers  (2026-03-18T18:14:34Z)
• Foundation CAN LM: A Pretrained Language Model For Automotive CAN Data [1.1091582432763738]
  Controller Area Network (CAN) bus provides rich source of vehicular signals for applications in automotive and auto insurance domains.<n>Existing pipelines largely train isolated task-specific models on raw CAN data.<n>We introduce the foundation CAN model paradigm: large-scale pretraining followed by task-specific adaptation.
  arXiv  Detail & Related papers  (2026-01-31T19:00:48Z)
• CANTXSec: A Deterministic Intrusion Detection and Prevention System for CAN Bus Monitoring ECU Activations [53.036288487863786]
  We propose CANTXSec, the first deterministic Intrusion Detection and Prevention system based on physical ECU activations.<n>It detects and prevents classical attacks in the CAN bus, while detecting advanced attacks that have been less investigated in the literature.<n>We prove the effectiveness of our solution on a physical testbed, where we achieve 100% detection accuracy in both classes of attacks while preventing 100% of FIAs.
  arXiv  Detail & Related papers  (2025-05-14T13:37:07Z)
• Utilizing Background Knowledge for Robust Reasoning over Traffic Situations [63.45021731775964]
  We focus on a complementary research aspect of Intelligent Transportation: traffic understanding. We scope our study to text-based methods and datasets given the abundant commonsense knowledge. We adopt three knowledge-driven approaches for zero-shot QA over traffic situations.
  arXiv  Detail & Related papers  (2022-12-04T09:17:24Z)
• STC-IDS: Spatial-Temporal Correlation Feature Analyzing based Intrusion Detection System for Intelligent Connected Vehicles [7.301018758489822]
  We present a novel model for automotive intrusion detection by spatial-temporal correlation features of in-vehicle communication traffic (STC-IDS) Specifically, the proposed model exploits an encoding-detection architecture. In the encoder part, spatial and temporal relations are encoded simultaneously. The encoded information is then passed to the detector for generating forceful spatial-temporal attention features and enabling anomaly classification.
  arXiv  Detail & Related papers  (2022-04-23T04:22:58Z)
• Detecting CAN Masquerade Attacks with Signal Clustering Similarity [2.2881898195409884]
  Fabrication attacks are the easiest to administer and the easiest to detect because they disrupt frame frequency. masquerade attacks can be detected by computing time series clustering similarity using hierarchical clustering on the vehicle's CAN signals. We develop a forensic tool as a proof of concept to demonstrate the potential of the proposed approach for detecting CAN masquerade attacks.
  arXiv  Detail & Related papers  (2022-01-07T20:25:40Z)
• Counterfactual Variable Control for Robust and Interpretable Question Answering [57.25261576239862]
  Deep neural network based question answering (QA) models are neither robust nor explainable in many cases. In this paper, we inspect such spurious "capability" of QA models using causal inference. We propose a novel approach called Counterfactual Variable Control (CVC) that explicitly mitigates any shortcut correlation.
  arXiv  Detail & Related papers  (2020-10-12T10:09:05Z)
• Generating Diverse and Consistent QA pairs from Contexts with Information-Maximizing Hierarchical Conditional VAEs [62.71505254770827]
  We propose a conditional variational autoencoder (HCVAE) for generating QA pairs given unstructured texts as contexts. Our model obtains impressive performance gains over all baselines on both tasks, using only a fraction of data for training.
  arXiv  Detail & Related papers  (2020-05-28T08:26:06Z)
• Robust Question Answering Through Sub-part Alignment [53.94003466761305]
  We model question answering as an alignment problem. We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets.
  arXiv  Detail & Related papers  (2020-04-30T09:10:57Z)
• Conditional Self-Attention for Query-based Summarization [49.616774159367516]
  We propose textitconditional self-attention (CSA), a neural network module designed for conditional dependency modeling. Experiments on Debatepedia and HotpotQA benchmark datasets show CSA consistently outperforms vanilla Transformer.
  arXiv  Detail & Related papers  (2020-02-18T02:22:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.