Interactive System-wise Anomaly Detection
- URL: http://arxiv.org/abs/2304.10704v1
- Date: Fri, 21 Apr 2023 02:20:24 GMT
- Title: Interactive System-wise Anomaly Detection
- Authors: Guanchu Wang and Ninghao Liu and Daochen Zha and Xia Hu
- Abstract summary: Anomaly detection plays a fundamental role in various applications.
It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data.
We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
- Score: 66.3766756452743
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Anomaly detection, where data instances are discovered containing feature
patterns different from the majority, plays a fundamental role in various
applications. However, it is challenging for existing methods to handle the
scenarios where the instances are systems whose characteristics are not readily
observed as data. Appropriate interactions are needed to interact with the
systems and identify those with abnormal responses. Detecting system-wise
anomalies is a challenging task due to several reasons including: how to
formally define the system-wise anomaly detection problem; how to find the
effective activation signal for interacting with systems to progressively
collect the data and learn the detector; how to guarantee stable training in
such a non-stationary scenario with real-time interactions? To address the
challenges, we propose InterSAD (Interactive System-wise Anomaly Detection).
Specifically, first, we adopt Markov decision process to model the interactive
systems, and define anomalous systems as anomalous transition and anomalous
reward systems. Then, we develop an end-to-end approach which includes an
encoder-decoder module that learns system embeddings, and a policy network to
generate effective activation for separating embeddings of normal and anomaly
systems. Finally, we design a training method to stabilize the learning
process, which includes a replay buffer to store historical interaction data
and allow them to be re-sampled. Experiments on two benchmark environments,
including identifying the anomalous robotic systems and detecting user data
poisoning in recommendation models, demonstrate the superiority of InterSAD
compared with state-of-the-art baselines methods.
Related papers
- Heterogeneous Anomaly Detection for Software Systems via Semi-supervised
Cross-modal Attention [29.654681594903114]
We propose Hades, the first end-to-end semi-supervised approach to identify system anomalies based on heterogeneous data.
Our approach employs a hierarchical architecture to learn a global representation of the system status by fusing log semantics and metric patterns.
We evaluate Hades extensively on large-scale simulated data and datasets from Huawei Cloud.
arXiv Detail & Related papers (2023-02-14T09:02:11Z) - Quality-Based Conditional Processing in Multi-Biometrics: Application to
Sensor Interoperability [63.05238390013457]
We describe and evaluate the ATVS-UAM fusion approach submitted to the quality-based evaluation of the 2007 BioSecure Multimodal Evaluation Campaign.
Our approach is based on linear logistic regression, in which fused scores tend to be log-likelihood-ratios.
Results show that the proposed approach outperforms all the rule-based fusion schemes.
arXiv Detail & Related papers (2022-11-24T12:11:22Z) - Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data.
We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism.
We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
arXiv Detail & Related papers (2022-06-30T06:00:13Z) - Intrinsic Anomaly Detection for Multi-Variate Time Series [33.199682596741276]
Intrinsic anomalies are changes in the functional dependency structure between time series that represent an environment and time series that represent the internal state of a system that is placed in said environment.
These address the short-comings of existing anomaly detection methods that cannot differentiate between expected changes in the system's state and unexpected ones, i.e., changes in the system that deviate from the environment's influence.
Our most promising approach is fully unsupervised and combines adversarial learning and time series representation learning, thereby addressing problems such as label sparsity and subjectivity.
arXiv Detail & Related papers (2022-06-29T00:51:44Z) - Data-driven Residual Generation for Early Fault Detection with Limited
Data [4.129225533930966]
In many complex systems it is not feasible to develop highly accurate models for the systems.
Data-driven solutions have received an immense attention in the industries systems for several practical reasons.
Unlike the model-based methods it is straight forward to combine time series measurements such as pressure and voltage with other sources of information.
arXiv Detail & Related papers (2021-09-28T03:18:03Z) - A Novel Anomaly Detection Algorithm for Hybrid Production Systems based
on Deep Learning and Timed Automata [73.38551379469533]
DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems.
It combines deep learning and timed automata for creating behavioral model from observations.
The algorithm has been applied to few data sets including two from real systems and has shown promising results.
arXiv Detail & Related papers (2020-10-29T08:27:43Z) - Active Learning for Nonlinear System Identification with Guarantees [102.43355665393067]
We study a class of nonlinear dynamical systems whose state transitions depend linearly on a known feature embedding of state-action pairs.
We propose an active learning approach that achieves this by repeating three steps: trajectory planning, trajectory tracking, and re-estimation of the system from all available data.
We show that our method estimates nonlinear dynamical systems at a parametric rate, similar to the statistical rate of standard linear regression.
arXiv Detail & Related papers (2020-06-18T04:54:11Z) - End-to-End Models for the Analysis of System 1 and System 2 Interactions
based on Eye-Tracking Data [99.00520068425759]
We propose a computational method, within a modified visual version of the well-known Stroop test, for the identification of different tasks and potential conflicts events.
A statistical analysis shows that the selected variables can characterize the variation of attentive load within different scenarios.
We show that Machine Learning techniques allow to distinguish between different tasks with a good classification accuracy.
arXiv Detail & Related papers (2020-02-03T17:46:13Z) - Counter-example Guided Learning of Bounds on Environment Behavior [11.357397596759172]
We present a data-driven solution that allows for a system to be evaluated for specification conformance without an accurate model of the environment.
Our approach involves learning a conservative reactive bound of the environment's behavior using data and specification of the system's desired behavior.
arXiv Detail & Related papers (2020-01-20T19:58:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.