AMAD: AutoMasked Attention for Unsupervised Multivariate Time Series Anomaly Detection

• URL: http://arxiv.org/abs/2504.06643v3
• Date: Fri, 25 Apr 2025 15:30:42 GMT
• Title: AMAD: AutoMasked Attention for Unsupervised Multivariate Time Series Anomaly Detection
• Authors: Tiange Huang, Yongjun Li,
• Abstract summary: AMAD integrates textbfAutotextbfMasked Attention for UMTStextbfAD scenarios.<n>AMAD provides a robust and adaptable solution to UMTSAD challenges.
• Score: 0.7371521417300614
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Unsupervised multivariate time series anomaly detection (UMTSAD) plays a critical role in various domains, including finance, networks, and sensor systems. In recent years, due to the outstanding performance of deep learning in general sequential tasks, many models have been specialized for deep UMTSAD tasks and have achieved impressive results, particularly those based on the Transformer and self-attention mechanisms. However, the sequence anomaly association assumptions underlying these models are often limited to specific predefined patterns and scenarios, such as concentrated or peak anomaly patterns. These limitations hinder their ability to generalize to diverse anomaly situations, especially where the lack of labels poses significant challenges. To address these issues, we propose AMAD, which integrates \textbf{A}uto\textbf{M}asked Attention for UMTS\textbf{AD} scenarios. AMAD introduces a novel structure based on the AutoMask mechanism and an attention mixup module, forming a simple yet generalized anomaly association representation framework. This framework is further enhanced by a Max-Min training strategy and a Local-Global contrastive learning approach. By combining multi-scale feature extraction with automatic relative association modeling, AMAD provides a robust and adaptable solution to UMTSAD challenges. Extensive experimental results demonstrate that the proposed model achieving competitive performance results compared to SOTA benchmarks across a variety of datasets.

Related papers

• See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers [23.701716999879636]
  Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data. We introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA) to enhance both the detection and interpretation of anomalies.
  arXiv  Detail & Related papers  (2024-11-04T10:28:41Z)
• DMAD: Dual Memory Bank for Real-World Anomaly Detection [90.97573828481832]
  We propose a new framework named Dual Memory bank enhanced representation learning for Anomaly Detection (DMAD) DMAD employs a dual memory bank to calculate feature distance and feature attention between normal and abnormal patterns. We evaluate DMAD on the MVTec-AD and VisA datasets.
  arXiv  Detail & Related papers  (2024-03-19T02:16:32Z)
• FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion [29.130355774088205]
  FuseMoE is a mixture-of-experts framework incorporated with an innovative gating function. Designed to integrate a diverse number of modalities, FuseMoE is effective in managing scenarios with missing modalities and irregularly sampled data trajectories.
  arXiv  Detail & Related papers  (2024-02-05T17:37:46Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• Beyond Sharing: Conflict-Aware Multivariate Time Series Anomaly Detection [18.796225184893874]
  We introduce CAD, a Conflict-aware Anomaly Detection algorithm. We find that the poor performance of vanilla MMoE mainly comes from the input-output misalignment settings of MTS formulation. We show that CAD obtains an average F1-score of 0.943 across three public datasets, notably outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-17T11:00:01Z)
• Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment [38.574204922793626]
  We present a further analysis of visual modality incompleteness, benchmarking latest MMEA models on our proposed dataset MMEA-UMVM. Our research indicates that, in the face of modality incompleteness, models succumb to overfitting the modality noise, and exhibit performance oscillations or declines at high rates of missing modality. We introduce UMAEA, a robust multi-modal entity alignment approach designed to tackle uncertainly missing and ambiguous visual modalities.
  arXiv  Detail & Related papers  (2023-07-30T12:16:49Z)
• Abnormal Event Detection via Hypergraph Contrastive Learning [54.80429341415227]
  Abnormal event detection plays an important role in many real applications. In this paper, we study the unsupervised abnormal event detection problem in Attributed Heterogeneous Information Network. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns.
  arXiv  Detail & Related papers  (2023-04-02T08:23:20Z)
• Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models [80.23791222509644]
  Inconsistent AI models are considered brittle and untrustworthy by human users. We find that state-of-the-art vision-language models suffer from a surprisingly high degree of inconsistent behavior across tasks. We propose a rank correlation-based auxiliary training objective, computed over large automatically created cross-task contrast sets.
  arXiv  Detail & Related papers  (2023-03-28T16:57:12Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Deep Federated Anomaly Detection for Multivariate Time Series Data [93.08977495974978]
  We present a Federated Exemplar-based Deep Neural Network (Fed-ExDNN) to conduct anomaly detection for multivariate time series data on different edge devices. We show that ExDNN and Fed-ExDNN can outperform state-of-the-art anomaly detection algorithms and federated learning techniques.
  arXiv  Detail & Related papers  (2022-05-09T05:06:58Z)
• Forecast-based Multi-aspect Framework for Multivariate Time-series Anomaly Detection [35.5539484341082]
  We present FMUAD - a Forecast-based, Multi-aspect, Unsupervised Anomaly Detection framework. It captures the signature traits of anomaly types - spatial change, temporal change and correlation change - with independent modules. Experiments show it consistently outperforms other state-of-the-art forecast-based anomaly detectors.
  arXiv  Detail & Related papers  (2022-01-13T05:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.