Decomposition-based multi-scale transformer framework for time series anomaly detection

• URL: http://arxiv.org/abs/2504.14206v1
• Date: Sat, 19 Apr 2025 06:47:38 GMT
• Title: Decomposition-based multi-scale transformer framework for time series anomaly detection
• Authors: Wenxin Zhang, Cuicui Luo,
• Abstract summary: We propose a transformer-based framework built on decomposition (TransDe) for time series anomaly detection.<n>A multi-scale patch-based transformer architecture is proposed to exploit the representative dependencies of each decomposed component of the time series.<n>A novel asynchronous loss function with a stop-gradient strategy is introduced to enhance the performance of TransDe effectively.
• Score: 0.9438207505148947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time series anomaly detection is crucial for maintaining stable systems. Existing methods face two main challenges. First, it is difficult to directly model the dependencies of diverse and complex patterns within the sequences. Second, many methods that optimize parameters using mean squared error struggle with noise in the time series, leading to performance deterioration. To address these challenges, we propose a transformer-based framework built on decomposition (TransDe) for multivariate time series anomaly detection. The key idea is to combine the strengths of time series decomposition and transformers to effectively learn the complex patterns in normal time series data. A multi-scale patch-based transformer architecture is proposed to exploit the representative dependencies of each decomposed component of the time series. Furthermore, a contrastive learn paradigm based on patch operation is proposed, which leverages KL divergence to align the positive pairs, namely the pure representations of normal patterns between different patch-level views. A novel asynchronous loss function with a stop-gradient strategy is further introduced to enhance the performance of TransDe effectively. It can avoid time-consuming and labor-intensive computation costs in the optimization process. Extensive experiments on five public datasets are conducted and TransDe shows superiority compared with twelve baselines in terms of F1 score. Our code is available at https://github.com/shaieesss/TransDe.

Related papers

• VISTA: Unsupervised 2D Temporal Dependency Representations for Time Series Anomaly Detection [42.694234312755285]
  Time Series Anomaly Detection (TSAD) is essential for uncovering rare and potentially harmful events in unlabeled time series data. We introduce VISTA, a training-free, unsupervised TSAD algorithm designed to overcome these challenges.
  arXiv  Detail & Related papers  (2025-04-03T11:20:49Z)
• Sentinel: Multi-Patch Transformer with Temporal and Channel Attention for Time Series Forecasting [48.52101281458809]
  Transformer-based time series forecasting has recently gained strong interest due to the ability of transformers to model sequential data. We propose Sentinel, a transformer-based architecture composed of an encoder able to extract contextual information from the channel dimension. We introduce a multi-patch attention mechanism, which leverages the patching process to structure the input sequence in a way that can be naturally integrated into the transformer architecture.
  arXiv  Detail & Related papers  (2025-03-22T06:01:50Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies.<n>We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis.<n> Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• Sensorformer: Cross-patch attention with global-patch compression is effective for high-dimensional multivariate time series forecasting [12.103678233732584]
  We propose a new Transformer, Sensorformer, which first compresses the global patch information and then simultaneously extracts cross-variable and cross-time dependencies from the compressed representations.<n>Sensorformer can effectively capture the correct inter-variable correlations and causal relationships, even in the presence of dynamic causal lags between variables.
  arXiv  Detail & Related papers  (2025-01-06T03:14:47Z)
• Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching [46.58170057001437]
  We introduce the Rough Transformer, a variation of the Transformer model that operates on continuous-time representations of input sequences.<n>We find that, on a variety of time-series-related tasks, Rough Transformers consistently outperform their vanilla attention counterparts.
  arXiv  Detail & Related papers  (2024-05-31T14:00:44Z)
• Rough Transformers for Continuous and Efficient Time-Series Modelling [46.58170057001437]
  Time-series data in real-world medical settings typically exhibit long-range dependencies and are observed at non-uniform intervals. We introduce the Rough Transformer, a variation of the Transformer model which operates on continuous-time representations of input sequences. We find that Rough Transformers consistently outperform their vanilla attention counterparts while obtaining the benefits of Neural ODE-based models.
  arXiv  Detail & Related papers  (2024-03-15T13:29:45Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Compatible Transformer for Irregularly Sampled Multivariate Time Series [75.79309862085303]
  We propose a transformer-based encoder to achieve comprehensive temporal-interaction feature learning for each individual sample. We conduct extensive experiments on 3 real-world datasets and validate that the proposed CoFormer significantly and consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2023-10-17T06:29:09Z)
• FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification [53.55504611255664]
  FormerTime is a hierarchical representation model for improving the classification capacity for the multivariate time series classification task. It exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism.
  arXiv  Detail & Related papers  (2023-02-20T07:46:14Z)
• Are Transformers Effective for Time Series Forecasting? [13.268196448051308]
  Recently, there has been a surge of Transformer-based solutions for the time series forecasting (TSF) task. This study investigates whether Transformer-based techniques are the right solutions for long-term time series forecasting. We find that the relatively higher long-term forecasting accuracy of Transformer-based solutions has little to do with the temporal relation extraction capabilities of the Transformer architecture.
  arXiv  Detail & Related papers  (2022-05-26T17:17:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.