Partially Observable Online Change Detection via Smooth-Sparse
Decomposition
- URL: http://arxiv.org/abs/2009.10645v1
- Date: Tue, 22 Sep 2020 16:03:04 GMT
- Title: Partially Observable Online Change Detection via Smooth-Sparse
Decomposition
- Authors: Jie Guo, Hao Yan, Chen Zhang, Steven Hoi
- Abstract summary: We consider online change detection of high dimensional data streams with sparse changes, where only a subset of data streams can be observed at each sensing time point due to limited sensing capacities.
On the one hand, the detection scheme should be able to deal with partially observable data and meanwhile have efficient detection power for sparse changes.
In this paper, we propose a novel detection scheme called CDSSD. In particular, it describes the structure of high dimensional data with sparse changes by smooth-sparse decomposition.
- Score: 16.8028358824706
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We consider online change detection of high dimensional data streams with
sparse changes, where only a subset of data streams can be observed at each
sensing time point due to limited sensing capacities. On the one hand, the
detection scheme should be able to deal with partially observable data and
meanwhile have efficient detection power for sparse changes. On the other, the
scheme should be able to adaptively and actively select the most important
variables to observe to maximize the detection power. To address these two
points, in this paper, we propose a novel detection scheme called CDSSD. In
particular, it describes the structure of high dimensional data with sparse
changes by smooth-sparse decomposition, whose parameters can be learned via
spike-slab variational Bayesian inference. Then the posterior Bayes factor,
which incorporates the learned parameters and sparse change information, is
formulated as a detection statistic. Finally, by formulating the statistic as
the reward of a combinatorial multi-armed bandit problem, an adaptive sampling
strategy based on Thompson sampling is proposed. The efficacy and applicability
of our method in practice are demonstrated with numerical studies and a real
case study.
Related papers
- Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
We introduce an approach that is both general and parameter-efficient for face forgery detection.
We design a forgery-style mixture formulation that augments the diversity of forgery source domains.
We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
arXiv Detail & Related papers (2024-08-23T01:53:36Z) - Partially-Observable Sequential Change-Point Detection for Autocorrelated Data via Upper Confidence Region [12.645304808491309]
We propose a detection scheme called adaptive upper confidence region with state space model (AUCRSS) for sequential change point detection.
A partially-observable Kalman filter algorithm is developed for online inference of SSM, and accordingly, a change point detection scheme based on a generalized likelihood ratio test is analyzed.
arXiv Detail & Related papers (2024-03-30T02:32:53Z) - Online Change Points Detection for Linear Dynamical Systems with Finite
Sample Guarantees [1.6026317505839445]
We study the online change point detection problem for linear dynamical systems with unknown dynamics.
We develop a data-dependent threshold that can be used in our test that allows one to achieve a pre-specified upper bound on the probability of making a false alarm.
arXiv Detail & Related papers (2023-11-30T18:08:16Z) - Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations.
In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
arXiv Detail & Related papers (2023-08-31T05:05:53Z) - Low-rank extended Kalman filtering for online learning of neural
networks from streaming data [71.97861600347959]
We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream.
The method is based on the extended Kalman filter (EKF), but uses a novel low-rank plus diagonal decomposition of the posterior matrix.
In contrast to methods based on variational inference, our method is fully deterministic, and does not require step-size tuning.
arXiv Detail & Related papers (2023-05-31T03:48:49Z) - High dimensional change-point detection: a complete graph approach [0.0]
We propose a complete graph-based, change-point detection algorithm to detect change of mean and variance from low to high-dimensional online data.
Inspired by complete graph structure, we introduce graph-spanning ratios to map high-dimensional data into metrics.
Our approach has high detection power with small and multiple scanning window, which allows timely detection of change-point in the online setting.
arXiv Detail & Related papers (2022-03-16T15:59:20Z) - Bandit Quickest Changepoint Detection [55.855465482260165]
Continuous monitoring of every sensor can be expensive due to resource constraints.
We derive an information-theoretic lower bound on the detection delay for a general class of finitely parameterized probability distributions.
We propose a computationally efficient online sensing scheme, which seamlessly balances the need for exploration of different sensing options with exploitation of querying informative actions.
arXiv Detail & Related papers (2021-07-22T07:25:35Z) - Online Structural Change-point Detection of High-dimensional Streaming
Data via Dynamic Sparse Subspace Learning [9.050841801109332]
We propose a dynamic sparse subspace learning approach for online structural change-point detection of high-dimensional streaming data.
A novel multiple structural change-point model is proposed and the properties of the estimators are investigated.
An efficient Pruned Exact Linear Time based algorithm is proposed for online optimization and change-point detection.
arXiv Detail & Related papers (2020-09-24T14:16:18Z) - Change Point Detection in Time Series Data using Autoencoders with a
Time-Invariant Representation [69.34035527763916]
Change point detection (CPD) aims to locate abrupt property changes in time series data.
Recent CPD methods demonstrated the potential of using deep learning techniques, but often lack the ability to identify more subtle changes in the autocorrelation statistics of the signal.
We employ an autoencoder-based methodology with a novel loss function, through which the used autoencoders learn a partially time-invariant representation that is tailored for CPD.
arXiv Detail & Related papers (2020-08-21T15:03:21Z) - Spatially Adaptive Inference with Stochastic Feature Sampling and
Interpolation [72.40827239394565]
We propose to compute features only at sparsely sampled locations.
We then densely reconstruct the feature map with an efficient procedure.
The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
arXiv Detail & Related papers (2020-03-19T15:36:31Z) - High-dimensional, multiscale online changepoint detection [7.502070498889449]
We introduce a new method for high-dimensional, online changepoint detection in settings where a $p$- Gaussian data stream may undergo a change in mean.
The algorithm is online in the sense that both its storage requirements and worst-case computational complexity per new observation are independent of the number of previous observations.
Simulations confirm the practical effectiveness of our proposal, which is implemented in the R package 'ocd', and we also demonstrate its utility on a seismology data set.
arXiv Detail & Related papers (2020-03-07T21:54:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.