Pi-Transformer: A Physics-informed Attention Mechanism for Time Series Anomaly Detection

• URL: http://arxiv.org/abs/2509.19985v1
• Date: Wed, 24 Sep 2025 10:47:48 GMT
• Title: Pi-Transformer: A Physics-informed Attention Mechanism for Time Series Anomaly Detection
• Abstract summary: We present Pi-Transformer, a physics-informed transformer with two attention pathways.<n>The model combines an alignment-weighted reconstruction signal with a mismatch signal that highlights timing and phase disruptions.<n>Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies.
• Score: 3.8673630752805437
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Anomalies in multivariate time series often arise from temporal context and cross-channel coordination rather than isolated outliers. We present Pi-Transformer, a physics-informed transformer with two attention pathways: a data-driven series attention and a smoothly evolving prior attention that encodes temporal invariants such as scale-related self-similarity and phase synchrony. The prior acts as a stable reference that calibrates reconstruction error. During training, we pair a reconstruction objective with a divergence term that encourages agreement between the two attentions while keeping them meaningfully distinct; the prior is regularised to evolve smoothly and is lightly distilled towards dataset-level statistics. At inference, the model combines an alignment-weighted reconstruction signal (Energy) with a mismatch signal that highlights timing and phase disruptions, and fuses them into a single score for detection. Across five benchmarks (SMD, MSL, SMAP, SWaT, and PSM), Pi-Transformer achieves state-of-the-art or highly competitive F1, with particular strength on timing and phase-breaking anomalies. Case analyses show complementary behaviour of the two streams and interpretable detections around regime changes. Embedding physics-informed priors into attention yields a calibrated and robust approach to anomaly detection in complex multivariate systems. Code is publicly available at this GitHub repository\footnote{https://github.com/sepehr-m/Pi-Transformer}.

Related papers

• TIE: Time Interval Encoding for Video Generation over Events [50.66585165263848]
  Director-style prompting, robotic action prediction, and interactive video agents demand temporal grounding over concurrent events.<n>Modern video generators represent time as discrete points through point-wise positional encodings.<n>Time Interval TIE is a principled, plug-and-play interval-aware generalization.
  arXiv  Detail & Related papers  (2026-05-11T13:23:14Z)
• Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection [1.7751300245073598]
  AxonAD is an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process.<n>A tail-aggregated query mismatch score measures cosine deviation between predicted and target queries on recent timesteps.<n>On in-vehicle telemetry with interval annotations, AxonAD improves ranking quality and temporal localization over strong baselines.
  arXiv  Detail & Related papers  (2026-03-13T11:40:51Z)
• LEFT: Learnable Fusion of Tri-view Tokens for Unsupervised Time Series Anomaly Detection [53.191369031661885]
  Unsupervised time series anomaly detection aims to build a model for identifying abnormal timestamps without assuming the availability of annotations.<n>We present Learnable Fusion of Tri-view Tokens (LEFT), a unified unsupervised TSAD framework that models anomalies as inconsistencies across complementary representations.<n>Experiments on real-world benchmarks show that LEFT yields the best detection accuracy against SOTA baselines, while achieving a 5x reduction on FLOPs and 8x speed-up for training.
  arXiv  Detail & Related papers  (2026-02-09T13:33:49Z)
• DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters [50.43534351968113]
  Existing generative time series models do not address the multi-dimensional properties of time series data well.<n>Inspired by Multimodal Diffusion Transformers that integrate textual guidance into video generation, we propose Diffusion Transformers for Time Series (DiTS)
  arXiv  Detail & Related papers  (2026-02-06T10:48:13Z)
• Integrating Quantum-Classical Attention in Patch Transformers for Enhanced Time Series Forecasting [4.580983642743026]
  QCAAPatchTF is a quantum attention network integrated with an advanced patch-based transformer.<n> leveraging quantum superpositions, entanglement, and variational quantum eigensolver principles.<n>QCAAPatchTF achieves state-of-the-art performance in both long-term and short-term forecasting, classification, and anomaly detection tasks.
  arXiv  Detail & Related papers  (2025-03-31T17:23:36Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• A Novel Spatiotemporal Correlation Anomaly Detection Method Based on Time-Frequency-Domain Feature Fusion and a Dynamic Graph Neural Network in Wireless Sensor Network [9.031267813814118]
  Attention-based transformers have played an important role in wireless sensor network (WSN) timing anomaly detection due to their ability to capture long-term dependencies.<n>This paper proposes a WSN anomaly detection method that integrates frequency-domain features with dynamic graph neural networks (GNN) under a designed self-encoder reconstruction framework.
  arXiv  Detail & Related papers  (2025-02-25T04:34:18Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• iTransformer: Inverted Transformers Are Effective for Time Series Forecasting [62.40166958002558]
  We propose iTransformer, which simply applies the attention and feed-forward network on the inverted dimensions. The iTransformer model achieves state-of-the-art on challenging real-world datasets.
  arXiv  Detail & Related papers  (2023-10-10T13:44:09Z)
• CARD: Channel Aligned Robust Blend Transformer for Time Series Forecasting [50.23240107430597]
  We design a special Transformer, i.e., Channel Aligned Robust Blend Transformer (CARD for short), that addresses key shortcomings of CI type Transformer in time series forecasting. First, CARD introduces a channel-aligned attention structure that allows it to capture both temporal correlations among signals. Second, in order to efficiently utilize the multi-scale knowledge, we design a token blend module to generate tokens with different resolutions. Third, we introduce a robust loss function for time series forecasting to alleviate the potential overfitting issue.
  arXiv  Detail & Related papers  (2023-05-20T05:16:31Z)
• AnomalyBERT: Self-Supervised Transformer for Time Series Anomaly Detection using Data Degradation Scheme [0.7216399430290167]
  Anomaly detection task for time series, especially for unlabeled data, has been a challenging problem. We address it by applying a suitable data degradation scheme to self-supervised model training. Inspired by the self-attention mechanism, we design a Transformer-based architecture to recognize the temporal context.
  arXiv  Detail & Related papers  (2023-05-08T05:42:24Z)
• DRAformer: Differentially Reconstructed Attention Transformer for Time-Series Forecasting [7.805077630467324]
  Time-series forecasting plays an important role in many real-world scenarios, such as equipment life cycle forecasting, weather forecasting, and traffic flow forecasting. It can be observed from recent research that a variety of transformer-based models have shown remarkable results in time-series forecasting. However, there are still some issues that limit the ability of transformer-based models on time-series forecasting tasks.
  arXiv  Detail & Related papers  (2022-06-11T10:34:29Z)
• A Differential Attention Fusion Model Based on Transformer for Time Series Forecasting [4.666618110838523]
  Time series forecasting is widely used in the fields of equipment life cycle forecasting, weather forecasting, traffic flow forecasting, and other fields. Some scholars have tried to apply Transformer to time series forecasting because of its powerful parallel training ability. The existing Transformer methods do not pay enough attention to the small time segments that play a decisive role in prediction.
  arXiv  Detail & Related papers  (2022-02-23T10:33:12Z)
• Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT [11.480824844205864]
  This work proposed a novel framework, namely GTA, for multivariate time series anomaly detection by automatically learning a graph structure followed by the graph convolution. We also devised a novel graph convolution named Influence propagation convolution to model the anomaly information flow between graph nodes. The experiments on four public anomaly detection benchmarks further demonstrate our approach's superiority over other state-of-the-arts.
  arXiv  Detail & Related papers  (2021-04-08T01:45:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.