mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale

• URL: http://arxiv.org/abs/2506.21550v1
• Date: Thu, 26 Jun 2025 17:59:58 GMT
• Title: mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale
• Authors: Xiaona Zhou, Constantin Brif, Ismini Lourentzou,
• Abstract summary: mTSBench is the largest benchmark to date for MTS-AD and unsupervised model selection.<n>It spans 344 labeled time series across 19 datasets and 12 diverse application domains.
• Score: 2.610026343726206
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multivariate time series anomaly detection (MTS-AD) is critical in domains like healthcare, cybersecurity, and industrial monitoring, yet remains challenging due to complex inter-variable dependencies, temporal dynamics, and sparse anomaly labels. We introduce mTSBench, the largest benchmark to date for MTS-AD and unsupervised model selection, spanning 344 labeled time series across 19 datasets and 12 diverse application domains. mTSBench evaluates 24 anomaly detection methods, including large language model (LLM)-based detectors for multivariate time series, and systematically benchmarks unsupervised model selection techniques under standardized conditions. Consistent with prior findings, our results confirm that no single detector excels across datasets, underscoring the importance of model selection. However, even state-of-the-art selection methods remain far from optimal, revealing critical gaps. mTSBench provides a unified evaluation suite to enable rigorous, reproducible comparisons and catalyze future advances in adaptive anomaly detection and robust model selection.

Related papers

• RAMSeS: Robust and Adaptive Model Selection for Time-Series Anomaly Detection Algorithms [4.651327752886104]
  Time-series data vary widely across domains, making a universal anomaly detector impractical.<n>We present the Robust and Adaptive Model Selection for Time-Series Anomaly Detection RAMSeS framework.
  arXiv  Detail & Related papers  (2026-02-25T10:36:32Z)
• Contextual and Seasonal LSTMs for Time Series Anomaly Detection [49.50689313712684]
  We propose a novel prediction-based framework named Contextual and Seasonal LSTMs (CS-LSTMs)<n>CS-LSTMs are built upon a noise decomposition strategy and jointly leverage contextual dependencies and seasonal patterns.<n>They consistently outperform state-of-the-art methods, highlighting their effectiveness and practical value in robust time series anomaly detection.
  arXiv  Detail & Related papers  (2026-02-10T11:46:15Z)
• Multivariate Time-series Anomaly Detection via Dynamic Model Pool & Ensembling [10.831568759799657]
  We propose DMPEAD, a Dynamic Model Pool and Ensembling framework for MTS Anomaly Detection.<n>We show that our model outperforms all baselines, demonstrating superior adaptability and scalability.
  arXiv  Detail & Related papers  (2026-01-05T11:48:21Z)
• FusAD: Time-Frequency Fusion with Adaptive Denoising for General Time Series Analysis [92.23551599659186]
  Time series analysis plays a vital role in fields such as finance, healthcare, industry, and meteorology.<n>FusAD is a unified analysis framework designed for diverse time series tasks.
  arXiv  Detail & Related papers  (2025-12-16T04:34:27Z)
• MSAD: A Deep Dive into Model Selection for Time series Anomaly Detection [13.15424393895753]
  Anomaly detection is a fundamental task for time series analytics.<n>No overall best anomaly detection methods exist when applied to very heterogeneous time series.<n>Existing AutoML solutions are, unfortunately, not directly applicable to time series anomaly detection.
  arXiv  Detail & Related papers  (2025-10-30T16:09:51Z)
• Time-RA: Towards Time Series Reasoning for Anomaly with LLM Feedback [55.284574165467525]
  Time-series Reasoning for Anomaly (Time-RA) transforms classical time series anomaly detection into a generative, reasoning-intensive task.<n>Also, we introduce the first real-world multimodal benchmark dataset, RATs40K, explicitly annotated for anomaly reasoning.
  arXiv  Detail & Related papers  (2025-07-20T18:02:50Z)
• See it, Think it, Sorted: Large Multimodal Models are Few-shot Time Series Anomaly Analyzers [23.701716999879636]
  Time series anomaly detection (TSAD) is becoming increasingly vital due to the rapid growth of time series data. We introduce a pioneering framework called the Time Series Anomaly Multimodal Analyzer (TAMA) to enhance both the detection and interpretation of anomalies.
  arXiv  Detail & Related papers  (2024-11-04T10:28:41Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalities [55.87169702896249]
  Unsupervised Domain Adaptation (DA) consists of adapting a model trained on a labeled source domain to perform well on an unlabeled target domain with some data distribution shift.<n>We present a complete and fair evaluation of existing shallow algorithms, including reweighting, mapping, and subspace alignment.<n>Our benchmark highlights the importance of realistic validation and provides practical guidance for real-life applications.
  arXiv  Detail & Related papers  (2024-07-16T12:52:29Z)
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
  We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T14:39:28Z)
• HCL-MTSAD: Hierarchical Contrastive Consistency Learning for Accurate Detection of Industrial Multivariate Time Series Anomalies [4.806959791183183]
  We propose a novel self-supervised hierarchical contrastive consistency learning method for detecting anomalies in industrial MTS. By developing a multi-layer contrastive loss, HCL-MTSAD can extensively mine data consistency and timestamp-temporal association. Experiments conducted on six diverse MTS retrieved from real cyber-physical systems and server machines, indicate that HCL-MTSAD's anomaly detection capability outperforms the state-of-the-art benchmark models by an average of 1.8% in terms of F1 score.
  arXiv  Detail & Related papers  (2024-04-12T03:39:33Z)
• TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models [21.658019069964755]
  Time series anomaly detection (TSAD) has gained significant attention due to its real-world applications. There is no effective way to verify whether TSAD can meet the requirements for real-world deployment. We propose an industrial-grade benchmark TimeSeriesBench to assess the performance of existing algorithms.
  arXiv  Detail & Related papers  (2024-02-16T16:25:20Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series [7.675917669905486]
  This paper presents a systematic and comprehensive evaluation of unsupervised and semi-supervised deep-learning based methods for anomaly detection and diagnosis. We vary the model and post-processing of model errors, through a grid of 10 models and 4 scoring functions, comparing these variants to state of the art methods. We find that the existing evaluation metrics either do not take events into account, or cannot distinguish between a good detector and trivial detectors.
  arXiv  Detail & Related papers  (2021-09-23T15:14:24Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• Automated Model Selection for Time-Series Anomaly Detection [6.396011708581161]
  Many companies need to monitor thousands of temporal signals for their applications and services and require instant feedback and alerts for potential incidents in time. The task is challenging because of the complex characteristics of time-series, which are messy, and often without proper labels. This prohibits training supervised models because of lack of labels and a single model hardly fits different time series. We present an automated model selection framework to automatically find the most suitable detection model with proper parameters for the incoming data.
  arXiv  Detail & Related papers  (2020-08-25T07:23:43Z)
• Supervised Feature Subset Selection and Feature Ranking for Multivariate Time Series without Feature Extraction [78.84356269545157]
  We introduce supervised feature ranking and feature subset selection algorithms for MTS classification. Unlike most existing supervised/unsupervised feature selection algorithms for MTS our techniques do not require a feature extraction step to generate a one-dimensional feature vector from the time series.
  arXiv  Detail & Related papers  (2020-05-01T07:46:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.