Related papers: Anomaly Detection for Aggregated Data Using Multi-Graph Autoencoder

Anomaly Detection for Aggregated Data Using Multi-Graph Autoencoder

URL: http://arxiv.org/abs/2101.04053v1
Date: Mon, 11 Jan 2021 17:38:42 GMT
Title: Anomaly Detection for Aggregated Data Using Multi-Graph Autoencoder
Authors: Tomer Meirman, Roni Stern, Gilad Katz
Abstract summary: We focus on creating an Anomaly detection models for system logs. We present a thorough analysis of the aggregated data and the relationships between aggregated events. We propose Multiple-graphs autoencoder MGAE, a novel convolutional graphs-autoencoder model.
Score: 21.81622481466591
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: In data systems, activities or events are continuously collected in the field to trace their proper executions. Logging, which means recording sequences of events, can be used for analyzing system failures and malfunctions, and identifying the causes and locations of such issues. In our research we focus on creating an Anomaly detection models for system logs. The task of anomaly detection is identifying unexpected events in dataset, which differ from the normal behavior. Anomaly detection models also assist in data systems analysis tasks. Modern systems may produce such a large amount of events monitoring every individual event is not feasible. In such cases, the events are often aggregated over a fixed period of time, reporting the number of times every event has occurred in that time period. This aggregation facilitates scaling, but requires a different approach for anomaly detection. In this research, we present a thorough analysis of the aggregated data and the relationships between aggregated events. Based on the initial phase of our research we present graphs representations of our aggregated dataset, which represent the different relationships between aggregated instances in the same context. Using the graph representation, we propose Multiple-graphs autoencoder MGAE, a novel convolutional graphs-autoencoder model which exploits the relationships of the aggregated instances in our unique dataset. MGAE outperforms standard graph-autoencoder models and the different experiments. With our novel MGAE we present 60% decrease in reconstruction error in comparison to standard graph autoencoder, which is expressed in reconstructing high-degree relationships.