Nonparametric and Online Change Detection in Multivariate Datastreams using QuantTree

• URL: http://arxiv.org/abs/2208.14801v1
• Date: Tue, 30 Aug 2022 07:24:16 GMT
• Title: Nonparametric and Online Change Detection in Multivariate Datastreams using QuantTree
• Abstract summary: We introduce QT-EWMA, a nonparametric change-detection algorithm that can control the expected time before a false alarm. We show that QT-EWMA and QT-EWMA-update control the false alarm rate better than state-of-the-art methods operating in similar conditions.
• Score: 7.085868567930685
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We address the problem of online change detection in multivariate datastreams, and we introduce QuantTree Exponentially Weighted Moving Average (QT-EWMA), a nonparametric change-detection algorithm that can control the expected time before a false alarm, yielding a desired Average Run Length (ARL$_0$). Controlling false alarms is crucial in many applications and is rarely guaranteed by online change-detection algorithms that can monitor multivariate datastreams without knowing the data distribution. Like many change-detection algorithms, QT-EWMA builds a model of the data distribution, in our case a QuantTree histogram, from a stationary training set. To monitor datastreams even when the training set is extremely small, we propose QT-EWMA-update, which incrementally updates the QuantTree histogram during monitoring, always keeping the ARL$_0$ under control. Our experiments, performed on synthetic and real-world datastreams, demonstrate that QT-EWMA and QT-EWMA-update control the ARL$_0$ and the false alarm rate better than state-of-the-art methods operating in similar conditions, achieving lower or comparable detection delays.

Related papers

• Continuous Wavelet Transform and Siamese Network-Based Anomaly Detection in Multi-variate Semiconductor Process Time Series [0.11184789007828977]
  anomaly prediction in semiconductor fabrication presents several critical challenges.<n>The paper presents a novel and generic approach for anomaly detection in MTS data using machine learning.<n>Our approach demonstrates high accuracy in identifying anomalies on a real FAB process time-series dataset.
  arXiv  Detail & Related papers  (2025-07-01T11:10:19Z)
• Change Detection in Multivariate data streams: Online Analysis with Kernel-QuantTree [4.492556918144639]
  Kernel-QuantTree Exponentially Weighted Moving Average (KQT-EWMA) Non-parametric change-detection algorithm that combines the Kernel-QuantTree (KQT) histogram and the EWMA statistic. Experiments on synthetic and real-world datasets demonstrate that KQT-EWMA can control $ARL_0$ while achieving detection delays comparable to or lower than state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-17T17:17:38Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• TeVAE: A Variational Autoencoder Approach for Discrete Online Anomaly Detection in Variable-state Multivariate Time-series Data [0.017476232824732776]
  We propose a temporal variational autoencoder (TeVAE) that can detect anomalies with minimal false positives when trained on unlabelled data. When properly configured, TeVAE flags anomalies only 6% of the time wrongly and detects 65% of anomalies present.
  arXiv  Detail & Related papers  (2024-07-09T13:32:33Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• A Robust and Explainable Data-Driven Anomaly Detection Approach For Power Electronics [56.86150790999639]
  We present two anomaly detection and classification approaches, namely the Matrix Profile algorithm and anomaly transformer. The Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. A series of custom filters is created and added to the detector to tune its sensitivity, recall, and detection accuracy.
  arXiv  Detail & Related papers  (2022-09-23T06:09:35Z)
• Autoencoder Based Iterative Modeling and Multivariate Time-Series Subsequence Clustering Algorithm [0.0]
  This paper introduces an algorithm for the detection of change-points and the identification of the corresponding subsequences in transient time-series data (MTSD) We use a recurrent neural network (RNN) based Autoencoder (AE) which is iteratively trained on incoming data. A model of the identified subsequence is saved and used for recognition of repeating subsequences as well as fast offline clustering.
  arXiv  Detail & Related papers  (2022-09-09T09:59:56Z)
• Contrastive predictive coding for Anomaly Detection in Multi-variate Time Series Data [6.463941665276371]
  We propose a Time-series Representational Learning through Contrastive Predictive Coding (TRL-CPC) towards anomaly detection in MVTS data. First, we jointly optimize an encoder, an auto-regressor and a non-linear transformation function to effectively learn the representations of the MVTS data sets.
  arXiv  Detail & Related papers  (2022-02-08T04:25:29Z)
• WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data [4.228718402877829]
  Change point detection methods have the ability to discover changes in an unsupervised fashion. We propose WATCH, a novel Wasserstein distance-based change point detection approach. An extensive evaluation shows that WATCH is capable of accurately identifying change points and outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-18T16:55:29Z)
• Wake Word Detection with Streaming Transformers [72.66551640048405]
  We show that our proposed Transformer model outperforms the baseline convolution network by 25% on average in false rejection rate at the same false alarm rate. Our experiments on the Mobvoi wake word dataset demonstrate that our proposed Transformer model outperforms the baseline convolution network by 25%.
  arXiv  Detail & Related papers  (2021-02-08T19:14:32Z)
• Contextual-Bandit Anomaly Detection for IoT Data in Distributed Hierarchical Edge Computing [65.78881372074983]
  IoT devices can hardly afford complex deep neural networks (DNN) models, and offloading anomaly detection tasks to the cloud incurs long delay. We propose and build a demo for an adaptive anomaly detection approach for distributed hierarchical edge computing (HEC) systems. We show that our proposed approach significantly reduces detection delay without sacrificing accuracy, as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-04-15T06:13:33Z)
• Adaptive Anomaly Detection for IoT Data in Hierarchical Edge Computing [71.86955275376604]
  We propose an adaptive anomaly detection approach for hierarchical edge computing (HEC) systems to solve this problem. We design an adaptive scheme to select one of the models based on the contextual information extracted from input data, to perform anomaly detection. We evaluate our proposed approach using a real IoT dataset, and demonstrate that it reduces detection delay by 84% while maintaining almost the same accuracy as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-01-10T05:29:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.