Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection

• URL: http://arxiv.org/abs/2603.12916v2
• Date: Mon, 16 Mar 2026 13:57:58 GMT
• Title: Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection
• Abstract summary: AxonAD is an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process.<n>A tail-aggregated query mismatch score measures cosine deviation between predicted and target queries on recent timesteps.<n>On in-vehicle telemetry with interval annotations, AxonAD improves ranking quality and temporal localization over strong baselines.
• Score: 1.7751300245073598
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multivariate time series anomalies often manifest as shifts in cross-channel dependencies rather than simple amplitude excursions. In autonomous driving, for instance, a steering command might be internally consistent but decouple from the resulting lateral acceleration. Residual-based detectors can miss such anomalies when flexible sequence models still reconstruct signals plausibly despite altered coordination. We introduce AxonAD, an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process. A gradient-updated reconstruction pathway is coupled with a history-only predictor that forecasts future query vectors from past context. This is trained via a masked predictor-target objective against an exponential moving average (EMA) target encoder. At inference, reconstruction error is combined with a tail-aggregated query mismatch score, which measures cosine deviation between predicted and target queries on recent timesteps. This dual approach provides sensitivity to structural dependency shifts while retaining amplitude-level detection. On proprietary in-vehicle telemetry with interval annotations and on the TSB-AD multi-variate suite (17 datasets, 180 series) with threshold-free and range-aware metrics, AxonAD improves ranking quality and temporal localization over strong baselines. Ablations confirm that query prediction and combined scoring are the primary drivers of the observed gains. Code is available at the URL https://github.com/iis-esslingen/AxonAD.

Related papers

• Real-Time Proactive Anomaly Detection via Forward and Backward Forecast Modeling [0.0]
  We introduce two proactive anomaly detection frameworks: the Forward Forecasting Model (FFM) and the Backward Reconstruction Model (BRM)<n>FFM forecasts future sequences to anticipate disruptions, while BRM reconstructs recent history from future context to uncover early precursors.<n>Our models support both continuous and discrete multivariate features, enabling robust performance in real-world settings.
  arXiv  Detail & Related papers  (2026-02-12T03:57:41Z)
• LEFT: Learnable Fusion of Tri-view Tokens for Unsupervised Time Series Anomaly Detection [53.191369031661885]
  Unsupervised time series anomaly detection aims to build a model for identifying abnormal timestamps without assuming the availability of annotations.<n>We present Learnable Fusion of Tri-view Tokens (LEFT), a unified unsupervised TSAD framework that models anomalies as inconsistencies across complementary representations.<n>Experiments on real-world benchmarks show that LEFT yields the best detection accuracy against SOTA baselines, while achieving a 5x reduction on FLOPs and 8x speed-up for training.
  arXiv  Detail & Related papers  (2026-02-09T13:33:49Z)
• SMKC: Sketch Based Kernel Correlation Images for Variable Cardinality Time Series Anomaly Detection [0.0]
  In operational environments, monitoring systems frequently experience sensor churn.<n>We propose SMKC, a framework that decouples the dynamic input structure from the anomaly detector.<n>We find that a detector using random projections and nearest neighbors on the SMKC representation performs competitively with fully trained baselines.
  arXiv  Detail & Related papers  (2026-01-28T21:15:11Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• Pi-Transformer: A Physics-informed Attention Mechanism for Time Series Anomaly Detection [3.8673630752805437]
  We present Pi-Transformer, a physics-informed transformer with two attention pathways.<n>The model combines an alignment-weighted reconstruction signal with a mismatch signal that highlights timing and phase disruptions.<n>Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies.
  arXiv  Detail & Related papers  (2025-09-24T10:47:48Z)
• FreDF: Learning to Forecast in the Frequency Domain [54.2091536822376]
  Time series modeling presents unique challenges due to autocorrelation in both historical data and future sequences.<n>We propose the Frequency-enhanced Direct Forecast (FreDF) which mitigates label autocorrelation by learning to forecast in the frequency domain.
  arXiv  Detail & Related papers  (2024-02-04T08:23:41Z)
• Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction [60.60223171143206]
  Trajectory prediction is a crucial undertaking in understanding entity movement or human behavior from observed sequences. Current methods often assume that the observed sequences are complete while ignoring the potential for missing values. This paper presents a unified framework, the Graph-based Conditional Variational Recurrent Neural Network (GC-VRNN), which can perform trajectory imputation and prediction simultaneously.
  arXiv  Detail & Related papers  (2023-03-28T14:27:27Z)
• AER: Auto-Encoder with Regression for Time Series Anomaly Detection [12.418290128163882]
  Anomaly detection on time series data is increasingly common across various industrial domains. Recent unsupervised machine learning methods have made remarkable progress in tackling this problem. We propose AER (Auto-encoder with Regression), a joint model that combines a vanilla auto-encoder and an LSTM regressor.
  arXiv  Detail & Related papers  (2022-12-27T17:22:21Z)
• Are we certain it's anomalous? [57.729669157989235]
  Anomaly detection in time series is a complex task since anomalies are rare due to highly non-linear temporal correlations. Here we propose the novel use of Hyperbolic uncertainty for Anomaly Detection (HypAD) HypAD learns self-supervisedly to reconstruct the input signal.
  arXiv  Detail & Related papers  (2022-11-16T21:31:39Z)
• Video Anomaly Detection via Prediction Network with Enhanced Spatio-Temporal Memory Exchange [21.334952965297667]
  Video anomaly detection is a challenging task because most anomalies are scarce and non-deterministic. We design a Convolutional LSTM Auto-Encoder prediction framework with enhanced large-temporal memory exchange. Evaluations on three popular benchmarks show that our framework outperforms existing prediction-based anomaly detection methods.
  arXiv  Detail & Related papers  (2022-06-26T16:10:56Z)
• Complex Event Forecasting with Prediction Suffix Trees: Extended Technical Report [70.7321040534471]
  Complex Event Recognition (CER) systems have become popular in the past two decades due to their ability to "instantly" detect patterns on real-time streams of events. There is a lack of methods for forecasting when a pattern might occur before such an occurrence is actually detected by a CER engine. We present a formal framework that attempts to address the issue of Complex Event Forecasting.
  arXiv  Detail & Related papers  (2021-09-01T09:52:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.