Partially-Observable Sequential Change-Point Detection for Autocorrelated Data via Upper Confidence Region

• URL: http://arxiv.org/abs/2404.00220v1
• Date: Sat, 30 Mar 2024 02:32:53 GMT
• Title: Partially-Observable Sequential Change-Point Detection for Autocorrelated Data via Upper Confidence Region
• Authors: Haijie Xu, Xiaochen Xian, Chen Zhang, Kaibo Liu,
• Abstract summary: We propose a detection scheme called adaptive upper confidence region with state space model (AUCRSS) for sequential change point detection. A partially-observable Kalman filter algorithm is developed for online inference of SSM, and accordingly, a change point detection scheme based on a generalized likelihood ratio test is analyzed.
• Score: 12.645304808491309
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sequential change point detection for multivariate autocorrelated data is a very common problem in practice. However, when the sensing resources are limited, only a subset of variables from the multivariate system can be observed at each sensing time point. This raises the problem of partially observable multi-sensor sequential change point detection. For it, we propose a detection scheme called adaptive upper confidence region with state space model (AUCRSS). It models multivariate time series via a state space model (SSM), and uses an adaptive sampling policy for efficient change point detection and localization. A partially-observable Kalman filter algorithm is developed for online inference of SSM, and accordingly, a change point detection scheme based on a generalized likelihood ratio test is developed. How its detection power relates to the adaptive sampling strategy is analyzed. Meanwhile, by treating the detection power as a reward, its connection with the online combinatorial multi-armed bandit (CMAB) problem is formulated and an adaptive upper confidence region algorithm is proposed for adaptive sampling policy design. Theoretical analysis of the asymptotic average detection delay is performed, and thorough numerical studies with synthetic data and real-world data are conducted to demonstrate the effectiveness of our method.

Related papers

• Reproduction of scan B-statistic for kernel change-point detection algorithm [10.49860279555873]
  Change-point detection has garnered significant attention due to its broad range of applications. In this paper, we reproduce a recently proposed online change-point detection algorithm based on an efficient kernel-based scan B-statistic. Our numerical experiments demonstrate that the scan B-statistic consistently delivers superior performance.
  arXiv  Detail & Related papers  (2024-08-23T15:12:31Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• Change-Point Detection in Industrial Data Streams based on Online Dynamic Mode Decomposition with Control [5.293458740536858]
  We propose a novel change-point detection method based on online Dynamic Mode Decomposition with control (ODMDwC) Our results demonstrate that this method yields intuitive and improved detection results compared to the Singular-Value-Decomposition-based method.
  arXiv  Detail & Related papers  (2024-07-08T14:18:33Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Adaptable and Interpretable Framework for Novelty Detection in Real-Time IoT Systems [0.609170287691728]
  RAID algorithm adapts to non-stationary effects such as data drift and change points that may not be accounted for during model development. RAID algorithm does not require changes to existing process automation infrastructures, making it highly deployable across different domains.
  arXiv  Detail & Related papers  (2023-04-06T09:16:37Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• Unsupervised Change Detection using DRE-CUSUM [14.73895038690252]
  DRE-CUSUM is an unsupervised density-ratio estimation (DRE) based approach to determine statistical changes in time-series data. We present a theoretical justification as well as accuracy guarantees which show that the proposed statistic can reliably detect statistical changes. We experimentally show the superiority of DRE-CUSUM using both synthetic and real-world datasets over existing state-of-the-art unsupervised algorithms.
  arXiv  Detail & Related papers  (2022-01-27T17:25:42Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Partially Observable Online Change Detection via Smooth-Sparse Decomposition [16.8028358824706]
  We consider online change detection of high dimensional data streams with sparse changes, where only a subset of data streams can be observed at each sensing time point due to limited sensing capacities. On the one hand, the detection scheme should be able to deal with partially observable data and meanwhile have efficient detection power for sparse changes. In this paper, we propose a novel detection scheme called CDSSD. In particular, it describes the structure of high dimensional data with sparse changes by smooth-sparse decomposition.
  arXiv  Detail & Related papers  (2020-09-22T16:03:04Z)
• Change Point Detection in Time Series Data using Autoencoders with a Time-Invariant Representation [69.34035527763916]
  Change point detection (CPD) aims to locate abrupt property changes in time series data. Recent CPD methods demonstrated the potential of using deep learning techniques, but often lack the ability to identify more subtle changes in the autocorrelation statistics of the signal. We employ an autoencoder-based methodology with a novel loss function, through which the used autoencoders learn a partially time-invariant representation that is tailored for CPD.
  arXiv  Detail & Related papers  (2020-08-21T15:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.