Temporal cross-validation impacts multivariate time series subsequence anomaly detection evaluation

• URL: http://arxiv.org/abs/2506.12183v1
• Date: Fri, 13 Jun 2025 19:14:44 GMT
• Title: Temporal cross-validation impacts multivariate time series subsequence anomaly detection evaluation
• Authors: Steven C. Hespeler, Pablo Moriano, Mingyan Li, Samuel C. Hollifield,
• Abstract summary: Time series cross-validation (TSCV) techniques aim to preserve temporal ordering during model evaluation.<n>This study systematically investigates the effect of TSCV strategy on the precision-recall characteristics of classifiers trained to detect fault-like anomalies in MTS datasets.
• Score: 3.43058724483837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating anomaly detection in multivariate time series (MTS) requires careful consideration of temporal dependencies, particularly when detecting subsequence anomalies common in fault detection scenarios. While time series cross-validation (TSCV) techniques aim to preserve temporal ordering during model evaluation, their impact on classifier performance remains underexplored. This study systematically investigates the effect of TSCV strategy on the precision-recall characteristics of classifiers trained to detect fault-like anomalies in MTS datasets. We compare walk-forward (WF) and sliding window (SW) methods across a range of validation partition configurations and classifier types, including shallow learners and deep learning (DL) classifiers. Results show that SW consistently yields higher median AUC-PR scores and reduced fold-to-fold performance variance, particularly for deep architectures sensitive to localized temporal continuity. Furthermore, we find that classifier generalization is sensitive to the number and structure of temporal partitions, with overlapping windows preserving fault signatures more effectively at lower fold counts. A classifier-level stratified analysis reveals that certain algorithms, such as random forests (RF), maintain stable performance across validation schemes, whereas others exhibit marked sensitivity. This study demonstrates that TSCV design in benchmarking anomaly detection models on streaming time series and provide guidance for selecting evaluation strategies in temporally structured learning environments.

Related papers

• FreRA: A Frequency-Refined Augmentation for Contrastive Learning on Time Series Classification [56.925103708982164]
  We present a novel perspective from the frequency domain and identify three advantages for downstream classification: global, independent, and compact.<n>We propose the lightweight yet effective Frequency Refined Augmentation (FreRA) tailored for time series contrastive learning on classification tasks.<n>FreRA consistently outperforms ten leading baselines on time series classification, anomaly detection, and transfer learning tasks.
  arXiv  Detail & Related papers  (2025-05-29T07:18:28Z)
• Multivariate Time Series Anomaly Detection by Capturing Coarse-Grained Intra- and Inter-Variate Dependencies [14.784236273395017]
  We introduce MtsCID, a novel semi-supervised multivariate time series anomaly detection method.<n>We show that MtsCID achieves performance comparable or superior to state-of-the-art benchmark methods.
  arXiv  Detail & Related papers  (2025-01-22T05:53:12Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Classification of Anomalies in Telecommunication Network KPI Time Series [1.2937020918620652]
  This paper proposes a modular anomaly classification framework for network anomalies. The framework assumes separate entities for the anomaly and the detector, allowing for a distinct treatment of anomaly detection and classification tasks on time series. This study has demonstrated the good performance of the anomaly classification models trained on simulated anomalies when applied to real-world network data.
  arXiv  Detail & Related papers  (2023-08-30T19:13:10Z)
• CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection [53.83593870825628]
  One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled time series in an unsupervised manner. We introduce a novel end-to-end self-supervised ContrAstive Representation Learning approach for time series anomaly detection.
  arXiv  Detail & Related papers  (2023-08-18T04:45:56Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series [7.675917669905486]
  This paper presents a systematic and comprehensive evaluation of unsupervised and semi-supervised deep-learning based methods for anomaly detection and diagnosis. We vary the model and post-processing of model errors, through a grid of 10 models and 4 scoring functions, comparing these variants to state of the art methods. We find that the existing evaluation metrics either do not take events into account, or cannot distinguish between a good detector and trivial detectors.
  arXiv  Detail & Related papers  (2021-09-23T15:14:24Z)
• Weakly Supervised Temporal Anomaly Segmentation with Dynamic Time Warping [23.829072352059953]
  We present WETAS, a novel framework that effectively identifies anomalous temporal segments in an input instance. We show that WETAS considerably outperforms other baselines in terms of the localization of temporal anomalies.
  arXiv  Detail & Related papers  (2021-08-15T21:22:06Z)
• Classification of multivariate weakly-labelled time-series with attention [0.0]
  Weakly labelled time-series are time-series containing noise and significant redundancies. This paper proposes an approach of exploiting context relevance of subsequences to improve classification accuracy.
  arXiv  Detail & Related papers  (2021-02-16T16:05:38Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• Change Point Detection in Time Series Data using Autoencoders with a Time-Invariant Representation [69.34035527763916]
  Change point detection (CPD) aims to locate abrupt property changes in time series data. Recent CPD methods demonstrated the potential of using deep learning techniques, but often lack the ability to identify more subtle changes in the autocorrelation statistics of the signal. We employ an autoencoder-based methodology with a novel loss function, through which the used autoencoders learn a partially time-invariant representation that is tailored for CPD.
  arXiv  Detail & Related papers  (2020-08-21T15:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.