Learning to Predict, Discover, and Reason in High-Dimensional Event Sequences

• URL: http://arxiv.org/abs/2603.16313v2
• Date: Thu, 19 Mar 2026 10:30:31 GMT
• Title: Learning to Predict, Discover, and Reason in High-Dimensional Event Sequences
• Abstract summary: In the automotive industry, domain experts manually group these codes into higher-level error patterns.<n>As vehicle complexity grows, this manual process becomes increasingly costly, error-prone, and difficult to scale.<n>This thesis addresses automated fault diagnostics by unifying event sequence modeling, causal discovery, and large language models.
• Score: 1.5229257192293202
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Electronic control units (ECUs) embedded within modern vehicles generate a large number of asynchronous events known as diagnostic trouble codes (DTCs). These discrete events form complex temporal sequences that reflect the evolving health of the vehicle's subsystems. In the automotive industry, domain experts manually group these codes into higher-level error patterns (EPs) using Boolean rules to characterize system faults and ensure safety. However, as vehicle complexity grows, this manual process becomes increasingly costly, error-prone, and difficult to scale. Notably, the number of unique DTCs in a modern vehicle is on the same order of magnitude as the vocabulary of a natural language, often numbering in the tens of thousands. This observation motivates a paradigm shift: treating diagnostic sequences as a language that can be modeled, predicted, and ultimately explained. Traditional statistical approaches fail to capture the rich dependencies and do not scale to high-dimensional datasets characterized by thousands of nodes, large sample sizes, and long sequence lengths. Specifically, the high cardinality of categorical event spaces in industrial logs poses a significant challenge, necessitating new machine learning architectures tailored to such event-driven systems. This thesis addresses automated fault diagnostics by unifying event sequence modeling, causal discovery, and large language models (LLMs) into a coherent framework for high-dimensional event streams. It is structured in three parts, reflecting a progressive transition from prediction to causal understanding and finally to reasoning for vehicle diagnostics. Consequently, we introduce several Transformer-based architectures for predictive maintenance, scalable sample- and population-level causal discovery frameworks and a multi-agent system that automates the synthesis of Boolean EP rules.

Related papers

• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers [57.54843029965778]
  Controllable pathology image synthesis requires reliable regulation of spatial layout, tissue morphology, and semantic detail.<n>We propose In-Context Diffusion Transformer (IC-DiT), a layout-aware generative model that incorporates spatial layouts, textual descriptions, and visual embeddings into a unified diffusion transformer.<n>IC-DiT achieves higher fidelity, stronger spatial controllability, and better diagnostic consistency than existing methods.
  arXiv  Detail & Related papers  (2026-03-11T06:14:11Z)
• Beyond Quantity: Trajectory Diversity Scaling for Code Agents [51.71414642763219]
  Trajectory Diversity Scaling is a data synthesis framework for code agents that scales performance through diversity rather than raw volume.<n> TDScaling integrates four innovations: (1) a Business Cluster mechanism that captures real-service logical dependencies; (2) a blueprint-driven multi-agent paradigm that enforces trajectory coherence; and (3) an adaptive evolution mechanism that steers toward long-tail scenarios.
  arXiv  Detail & Related papers  (2026-02-03T07:43:03Z)
• TRACE: Scalable Amortized Causal Discovery from Single Sequences via Autoregressive Density Estimation [14.409508347156397]
  We study causal discovery from a single observed sequence of discrete events generated by a process.<n>We introduce TRACE, a scalable framework that repurposes autoregressive models as pretrained density estimators for conditional mutual information estimation.
  arXiv  Detail & Related papers  (2026-02-01T10:18:27Z)
• Transforming Vehicle Diagnostics: A Multimodal Approach to Error Patterns Prediction [14.409508347156397]
  This paper presents BiCarFormer: the first multimodal approach to multi-label sequence classification of error codes into error patterns.<n>We show that BiCarFormer significantly improves classification performance compared to models that rely solely on DTC sequences and traditional sequence models.
  arXiv  Detail & Related papers  (2026-02-01T09:06:49Z)
• Variational decomposition autoencoding improves disentanglement of latent representations [14.3216921403324]
  We introduce variational decomposition autoencoding (VDA), a framework that extends VAEs by incorporating a strong structural bias toward signal decomposition.<n>VDA is instantiated through variational decomposition autoencoders (DecVAEs), i.e., encoder-only neural networks that combine a signal decomposition model, a contrastive self-supervised task, and variational prior approximation.<n>We demonstrate the effectiveness of DecVAEs on simulated data and three publicly available scientific datasets, spanning speech recognition, dysarthria severity evaluation, and emotional speech classification.
  arXiv  Detail & Related papers  (2026-01-11T10:16:34Z)
• Robust Group Anomaly Detection for Quasi-Periodic Network Time Series [47.60720976101336]
  We propose a framework to identify unusual and interesting time series within a network time series database.<n>We develop a surrogate-based optimization algorithm that can efficiently train the seq2GMM model.
  arXiv  Detail & Related papers  (2025-06-20T08:11:04Z)
• Spatial-Temporal-Spectral Unified Modeling for Remote Sensing Dense Prediction [20.1863553357121]
  Current deep learning architectures for remote sensing are fundamentally rigid.<n>We introduce the Spatial-Temporal-Spectral Unified Network (STSUN) for unified modeling.<n> STSUN can adapt to input and output data with arbitrary spatial sizes, temporal lengths, and spectral bands.<n>It unifies various dense prediction tasks and diverse semantic class predictions.
  arXiv  Detail & Related papers  (2025-05-18T07:39:17Z)
• MLAD: A Unified Model for Multi-system Log Anomaly Detection [35.68387377240593]
  We propose MLAD, a novel anomaly detection model that incorporates semantic relational reasoning across multiple systems. Specifically, we employ Sentence-bert to capture the similarities between log sequences and convert them into highly-dimensional learnable semantic vectors. We revamp the formulas of the Attention layer to discern the significance of each keyword in the sequence and model the overall distribution of the multi-system dataset.
  arXiv  Detail & Related papers  (2024-01-15T12:51:13Z)
• Causal Temporal Regime Structure Learning [49.77103348208835]
  We present CASTOR, a novel method that concurrently learns the Directed Acyclic Graph (DAG) for each regime.<n>We establish the identifiability of the regimes and DAGs within our framework.<n>Experiments show that CASTOR consistently outperforms existing causal discovery models.
  arXiv  Detail & Related papers  (2023-11-02T17:26:49Z)
• Self-awareness in intelligent vehicles: Feature based dynamic Bayesian models for abnormality detection [4.251384905163326]
  This paper aims to introduce a novel method to develop self-awareness in autonomous vehicles. Time-series data from the vehicles are used to develop the data-driven Dynamic Bayesian Network (DBN) models. An initial level collective awareness model that can perform joint anomaly detection in co-operative tasks is proposed.
  arXiv  Detail & Related papers  (2020-10-29T09:29:47Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers [42.93754828584075]
  We present a new Transformer architecture, Performer, based on Fast Attention Via Orthogonal Random features (FAVOR) Our mechanism scales linearly rather than quadratically in the number of tokens in the sequence, is characterized by sub-quadratic space complexity and does not incorporate any sparsity pattern priors. It provides strong theoretical guarantees: unbiased estimation of the attention matrix and uniform convergence.
  arXiv  Detail & Related papers  (2020-06-05T17:09:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.