AMAD: AutoMasked Attention for Unsupervised Multivariate Time Series Anomaly Detection

• URL: http://arxiv.org/abs/2504.06643v3
• Date: Fri, 25 Apr 2025 15:30:42 GMT
• Title: AMAD: AutoMasked Attention for Unsupervised Multivariate Time Series Anomaly Detection
• Abstract summary: AMAD integrates textbfAutotextbfMasked Attention for UMTStextbfAD scenarios.<n>AMAD provides a robust and adaptable solution to UMTSAD challenges.
• Score: 0.7371521417300614
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Unsupervised multivariate time series anomaly detection (UMTSAD) plays a critical role in various domains, including finance, networks, and sensor systems. In recent years, due to the outstanding performance of deep learning in general sequential tasks, many models have been specialized for deep UMTSAD tasks and have achieved impressive results, particularly those based on the Transformer and self-attention mechanisms. However, the sequence anomaly association assumptions underlying these models are often limited to specific predefined patterns and scenarios, such as concentrated or peak anomaly patterns. These limitations hinder their ability to generalize to diverse anomaly situations, especially where the lack of labels poses significant challenges. To address these issues, we propose AMAD, which integrates \textbf{A}uto\textbf{M}asked Attention for UMTS\textbf{AD} scenarios. AMAD introduces a novel structure based on the AutoMask mechanism and an attention mixup module, forming a simple yet generalized anomaly association representation framework. This framework is further enhanced by a Max-Min training strategy and a Local-Global contrastive learning approach. By combining multi-scale feature extraction with automatic relative association modeling, AMAD provides a robust and adaptable solution to UMTSAD challenges. Extensive experimental results demonstrate that the proposed model achieving competitive performance results compared to SOTA benchmarks across a variety of datasets.

Related papers

• VAN-AD: Visual Masked Autoencoder with Normalizing Flow For Time Series Anomaly Detection [17.972690703544764]
  Time series anomaly detection (TSAD) is essential for maintaining the reliability and security of IoT-enabled service systems.<n>In this paper, we investigate the applicability of large-scale vision models to TSAD.<n>We propose VAN-AD, a novel MAE-based framework for TSAD.
  arXiv  Detail & Related papers  (2026-03-27T08:48:59Z)
• Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection [12.745823680495887]
  We propose an unsupervised continual anomaly detection framework grounded in multimodal prompting.<n> Specifically, we introduce a Continual Multimodal Prompt Memory Bank (CMPMB) that progressively distills and retains normal patterns from both visual and textual domains.<n>We also devise a Defect-Semantic-Guided Adaptive Fusion Mechanism (DSG-AFM) that integrates an Adaptive Normalization Module (ANM) with a Dynamic Fusion Strategy (DFS) to jointly enhance detection accuracy and adversarial robustness.
  arXiv  Detail & Related papers  (2026-03-23T04:33:11Z)
• AlignMamba-2: Enhancing Multimodal Fusion and Sentiment Analysis with Modality-Aware Mamba [56.52470564147458]
  We propose textbfAlignMamba-2, an effective and efficient framework for multimodal fusion and sentiment analysis.<n>Our approach introduces a dual alignment strategy that regularizes the model using both Optimal Transport distance and Maximum Mean Discrepancy.<n>More importantly, we design a Modality-Aware Mamba layer, which employs a Mixture-of-Experts architecture with modality-specific and modality-shared experts.
  arXiv  Detail & Related papers  (2026-03-19T03:47:21Z)
• Towards One-for-All Anomaly Detection for Tabular Data [87.63505963517512]
  Tabular anomaly detection (TAD) is critical in many real-world applications.<n>We propose OFA-TAD, a framework that only requires one-time training on multiple source datasets.<n>Experiments on 34 datasets from 14 domains demonstrate that OFA-TAD achieves superior anomaly detection performance.
  arXiv  Detail & Related papers  (2026-03-15T14:42:05Z)
• AR-MOT: Autoregressive Multi-object Tracking [56.09738000988466]
  We propose a novel autoregressive paradigm that formulates MOT as a sequence generation task within a large language model (LLM) framework.<n>This design enables the model to output structured results through flexible sequence construction, without requiring any task-specific heads.<n>To enhance region-level visual perception, we introduce an Object Tokenizer based on a pretrained detector.
  arXiv  Detail & Related papers  (2026-01-05T09:17:28Z)
• FusAD: Time-Frequency Fusion with Adaptive Denoising for General Time Series Analysis [92.23551599659186]
  Time series analysis plays a vital role in fields such as finance, healthcare, industry, and meteorology.<n>FusAD is a unified analysis framework designed for diverse time series tasks.
  arXiv  Detail & Related papers  (2025-12-16T04:34:27Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Architecture Decoupling Is Not All You Need For Unified Multimodal Model [64.19284951218098]
  We propose Attention Interaction Alignment (AIA) loss, which explicitly learns Task-Specific multimodal interaction patterns during training.<n>AIA not only refines cross-modal attention patterns, but also boosts both generation and understanding performance.
  arXiv  Detail & Related papers  (2025-11-27T17:55:25Z)
• THEMIS: Unlocking Pretrained Knowledge with Foundation Model Embeddings for Anomaly Detection in Time Series [0.0]
  THEMIS is a new framework for time series anomaly detection that exploits pretrained knowledge from foundation models.<n>Our experiments show that this modular method achieves SOTA results on the MSL dataset and performs quite competitively on the SMAP and SWAT$*$ datasets.
  arXiv  Detail & Related papers  (2025-10-04T19:20:35Z)
• UniMMAD: Unified Multi-Modal and Multi-Class Anomaly Detection via MoE-Driven Feature Decompression [74.0893986012049]
  UniMMAD is a unified framework for multi-modal and multi-class anomaly detection.<n>UniMMAD achieves state-of-the-art performance on 9 anomaly detection datasets, spanning 3 fields, 12 modalities, and 66 classes.
  arXiv  Detail & Related papers  (2025-09-30T08:29:12Z)
• AdaMixT: Adaptive Weighted Mixture of Multi-Scale Expert Transformers for Time Series Forecasting [15.522567372502762]
  We propose a novel architecture named Adaptive Weighted Mixture of Multi-Scale Expert Transformers (AdaMixT)<n>AdaMixT introduces various patches and leverages both General Pre-trained Models (GPM) and Domain-specific Models (DSM) for multi-scale feature extraction.<n> Comprehensive experiments on eight widely used benchmarks, including Weather, Traffic, Electricity, ILI, and four ETT datasets, consistently demonstrate the effectiveness of AdaMixT.
  arXiv  Detail & Related papers  (2025-09-09T15:30:53Z)
• Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection [7.435598538875321]
  Video Anomaly Detection (VAD) is a challenging task due to the variability of anomalous events and the limited availability of labeled data.<n>We propose a novel framework that employs a set of expert models, each specialized in capturing specific anomaly types.<n>Our approach achieves state-of-the-art performance, with a 91.58% AUC on the UCF-Crime dataset, and demonstrates superior results on XD-Violence and MSAD datasets.
  arXiv  Detail & Related papers  (2025-08-08T13:48:48Z)
• Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection [53.137651284042434]
  Anomaly inspection plays a vital role in industrial manufacturing, but the scarcity of anomaly samples limits the effectiveness of existing methods.<n>We propose Generate grained Anomaly (GAA), a region-guided, few-shot anomaly image-mask pair generation framework.<n>GAA generates realistic, diverse, and semantically aligned anomalies using only a small number of samples.
  arXiv  Detail & Related papers  (2025-07-13T12:56:59Z)
• CLIP Meets Diffusion: A Synergistic Approach to Anomaly Detection [54.85000884785013]
  Anomaly detection is a complex problem due to the ambiguity in defining anomalies, the diversity of anomaly types, and the scarcity of training data.<n>We propose CLIPfusion, a method that leverages both discriminative and generative foundation models.<n>We believe that our method underscores the effectiveness of multi-modal and multi-model fusion in tackling the multifaceted challenges of anomaly detection.
  arXiv  Detail & Related papers  (2025-06-13T13:30:15Z)
• Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models [28.20124264650572]
  Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities across tasks.<n>They often exhibit difficulty in distinguishing task-relevant from irrelevant signals, particularly in tasks like Visual Question Answering (VQA)<n>This vulnerability becomes more evident in modality-specific tasks such as image classification or pure text question answering.<n>We propose a novel framework to fine-tune MLLMs, including perturbation-based data augmentation with both perturbations and adversarial perturbations.
  arXiv  Detail & Related papers  (2025-05-26T07:31:32Z)
• GenIAS: Generator for Instantiating Anomalies in time Series [54.959865643340535]
  We develop a generative model for time series anomaly detection (TSAD) using a variational autoencoder.<n>GenIAS is designed to produce diverse and realistic synthetic anomalies for TSAD tasks.<n>Our experiments demonstrate that GenIAS consistently outperforms seventeen traditional and deep anomaly detection models.
  arXiv  Detail & Related papers  (2025-02-12T10:10:04Z)
• See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers [23.701716999879636]
  Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data. We introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA) to enhance both the detection and interpretation of anomalies.
  arXiv  Detail & Related papers  (2024-11-04T10:28:41Z)
• DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series [61.91288852233078]
  In time series anomaly detection, the scarcity of labeled data poses a challenge to the development of accurate models.<n>We propose a novel Domain Contrastive learning model for Anomaly Detection in time series (DACAD)<n>Our model employs supervised contrastive loss for the source domain and self-supervised contrastive triplet loss for the target domain.
  arXiv  Detail & Related papers  (2024-04-17T11:20:14Z)
• DMAD: Dual Memory Bank for Real-World Anomaly Detection [90.97573828481832]
  We propose a new framework named Dual Memory bank enhanced representation learning for Anomaly Detection (DMAD) DMAD employs a dual memory bank to calculate feature distance and feature attention between normal and abnormal patterns. We evaluate DMAD on the MVTec-AD and VisA datasets.
  arXiv  Detail & Related papers  (2024-03-19T02:16:32Z)
• FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion [29.130355774088205]
  FuseMoE is a mixture-of-experts framework incorporated with an innovative gating function. Designed to integrate a diverse number of modalities, FuseMoE is effective in managing scenarios with missing modalities and irregularly sampled data trajectories.
  arXiv  Detail & Related papers  (2024-02-05T17:37:46Z)
• Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding [54.49214267905562]
  We construct a transformer-based framework for multi-modal manipulation detection and grounding tasks. Our framework simultaneously explores modality-specific features while preserving the capability for multi-modal alignment. We propose an implicit manipulation query (IMQ) that adaptively aggregates global contextual cues within each modality.
  arXiv  Detail & Related papers  (2023-09-22T06:55:41Z)
• Beyond Sharing: Conflict-Aware Multivariate Time Series Anomaly Detection [18.796225184893874]
  We introduce CAD, a Conflict-aware Anomaly Detection algorithm. We find that the poor performance of vanilla MMoE mainly comes from the input-output misalignment settings of MTS formulation. We show that CAD obtains an average F1-score of 0.943 across three public datasets, notably outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-17T11:00:01Z)
• Rethinking Uncertainly Missing and Ambiguous Visual Modality in Multi-Modal Entity Alignment [38.574204922793626]
  We present a further analysis of visual modality incompleteness, benchmarking latest MMEA models on our proposed dataset MMEA-UMVM. Our research indicates that, in the face of modality incompleteness, models succumb to overfitting the modality noise, and exhibit performance oscillations or declines at high rates of missing modality. We introduce UMAEA, a robust multi-modal entity alignment approach designed to tackle uncertainly missing and ambiguous visual modalities.
  arXiv  Detail & Related papers  (2023-07-30T12:16:49Z)
• Abnormal Event Detection via Hypergraph Contrastive Learning [54.80429341415227]
  Abnormal event detection plays an important role in many real applications. In this paper, we study the unsupervised abnormal event detection problem in Attributed Heterogeneous Information Network. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns.
  arXiv  Detail & Related papers  (2023-04-02T08:23:20Z)
• Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models [80.23791222509644]
  Inconsistent AI models are considered brittle and untrustworthy by human users. We find that state-of-the-art vision-language models suffer from a surprisingly high degree of inconsistent behavior across tasks. We propose a rank correlation-based auxiliary training objective, computed over large automatically created cross-task contrast sets.
  arXiv  Detail & Related papers  (2023-03-28T16:57:12Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Deep Federated Anomaly Detection for Multivariate Time Series Data [93.08977495974978]
  We present a Federated Exemplar-based Deep Neural Network (Fed-ExDNN) to conduct anomaly detection for multivariate time series data on different edge devices. We show that ExDNN and Fed-ExDNN can outperform state-of-the-art anomaly detection algorithms and federated learning techniques.
  arXiv  Detail & Related papers  (2022-05-09T05:06:58Z)
• Forecast-based Multi-aspect Framework for Multivariate Time-series Anomaly Detection [35.5539484341082]
  We present FMUAD - a Forecast-based, Multi-aspect, Unsupervised Anomaly Detection framework. It captures the signature traits of anomaly types - spatial change, temporal change and correlation change - with independent modules. Experiments show it consistently outperforms other state-of-the-art forecast-based anomaly detectors.
  arXiv  Detail & Related papers  (2022-01-13T05:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.