Precise Change Point Detection using Spectral Drift Detection

• URL: http://arxiv.org/abs/2205.06507v1
• Date: Fri, 13 May 2022 08:31:47 GMT
• Title: Precise Change Point Detection using Spectral Drift Detection
• Authors: Fabian Hinder, Andr\'e Artelt, Valerie Vaquet, Barbara Hammer
• Abstract summary: Concept drift refers to the phenomenon that the data generating distribution changes over time; as a consequence machine learning models may become inaccurate and need adjustment. In this paper we consider the problem of detecting those change points in unsupervised learning. We derive a new unsupervised drift detection algorithm, investigate its mathematical properties, and demonstrate its usefulness in several experiments.
• Score: 8.686667049158476
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The notion of concept drift refers to the phenomenon that the data generating distribution changes over time; as a consequence machine learning models may become inaccurate and need adjustment. In this paper we consider the problem of detecting those change points in unsupervised learning. Many unsupervised approaches rely on the discrepancy between the sample distributions of two time windows. This procedure is noisy for small windows, hence prone to induce false positives and not able to deal with more than one drift event in a window. In this paper we rely on structural properties of drift induced signals, which use spectral properties of kernel embedding of distributions. Based thereon we derive a new unsupervised drift detection algorithm, investigate its mathematical properties, and demonstrate its usefulness in several experiments.

Related papers

• Unsupervised Concept Drift Detection from Deep Learning Representations in Real-time [5.999777817331315]
  Concept Drift is a phenomenon in which the underlying data distribution and statistical properties of a target domain change over time. We propose DriftLens, an unsupervised real-time concept drift detection framework. It works on unstructured data by exploiting the distribution distances of deep learning representations.
  arXiv  Detail & Related papers  (2024-06-24T23:41:46Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Projection Regret: Reducing Background Bias for Novelty Detection via Diffusion Models [72.07462371883501]
  We propose emphProjection Regret (PR), an efficient novelty detection method that mitigates the bias of non-semantic information. PR computes the perceptual distance between the test image and its diffusion-based projection to detect abnormality. Extensive experiments demonstrate that PR outperforms the prior art of generative-model-based novelty detection methods by a significant margin.
  arXiv  Detail & Related papers  (2023-12-05T09:44:47Z)
• Class Distribution Monitoring for Concept Drift Detection [5.042611743157464]
  Class Distribution Monitoring (CDM) is an effective concept-drift detection scheme that monitors the class-conditional distributions of a datastream. We show that when the concept drift affects a few classes, CDM outperforms algorithms monitoring the overall data distribution. We also demonstrate that CDM inherits the properties of the underlying change detector, yielding an effective control over the expected time before a false alarm.
  arXiv  Detail & Related papers  (2022-10-16T07:15:05Z)
• Deep learning model solves change point detection for multiple change types [69.77452691994712]
  A change points detection aims to catch an abrupt disorder in data distribution. We propose an approach that works in the multiple-distributions scenario.
  arXiv  Detail & Related papers  (2022-04-15T09:44:21Z)
• Suitability of Different Metric Choices for Concept Drift Detection [9.76294323004155]
  Many unsupervised approaches for drift detection rely on measuring the discrepancy between the sample of two time windows. Most drift detection methods can be distinguished in what metric they use, how this metric is estimated, and how the decision threshold is found. We compare different types of estimators and metrics theoretically and empirically and investigate the relevance of the single metric components.
  arXiv  Detail & Related papers  (2022-02-19T01:11:32Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
  Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters. We show that our method improves both accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2021-09-27T01:09:08Z)
• Bayesian Autoencoders for Drift Detection in Industrial Environments [69.93875748095574]
  Autoencoders are unsupervised models which have been used for detecting anomalies in multi-sensor environments. Anomalies can come either from real changes in the environment (real drift) or from faulty sensory devices (virtual drift)
  arXiv  Detail & Related papers  (2021-07-28T10:19:58Z)
• Detecting Concept Drift With Neural Network Model Uncertainty [0.0]
  Uncertainty Drift Detection (UDD) is able to detect drifts without access to true labels. In contrast to input data-based drift detection, our approach considers the effects of the current input data on the properties of the prediction model. We show that UDD outperforms other state-of-the-art strategies on two synthetic as well as ten real-world data sets for both regression and classification tasks.
  arXiv  Detail & Related papers  (2021-07-05T08:56:36Z)
• Concept Drift Detection via Equal Intensity k-means Space Partitioning [40.77597229122878]
  Cluster-based histogram called equal intensity k-means space partitioning (EI-kMeans) Three algorithms are developed to implement concept drift detection, including a greedy centroids algorithm, a cluster amplify-shrink algorithm, and a drift detection algorithm. Experiments on synthetic and real-world datasets demonstrate the advantages of EI-kMeans and show its efficacy in detecting concept drift.
  arXiv  Detail & Related papers  (2020-04-24T08:00:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.