GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection

• URL: http://arxiv.org/abs/2309.05953v1
• Date: Tue, 12 Sep 2023 04:21:30 GMT
• Title: GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection
• Authors: Yufei Li, Yanchi Liu, Haoyu Wang, Zhengzhang Chen, Wei Cheng, Yuncong Chen, Wenchao Yu, Haifeng Chen, Cong Liu
• Abstract summary: GLAD is a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs. We introduce GLAD, a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs.
• Score: 49.9884374409624
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Logs play a crucial role in system monitoring and debugging by recording valuable system information, including events and states. Although various methods have been proposed to detect anomalies in log sequences, they often overlook the significance of considering relations among system components, such as services and users, which can be identified from log contents. Understanding these relations is vital for detecting anomalies and their underlying causes. To address this issue, we introduce GLAD, a Graph-based Log Anomaly Detection framework designed to detect relational anomalies in system logs. GLAD incorporates log semantics, relational patterns, and sequential patterns into a unified framework for anomaly detection. Specifically, GLAD first introduces a field extraction module that utilizes prompt-based few-shot learning to identify essential fields from log contents. Then GLAD constructs dynamic log graphs for sliding windows by interconnecting extracted fields and log events parsed from the log parser. These graphs represent events and fields as nodes and their relations as edges. Subsequently, GLAD utilizes a temporal-attentive graph edge anomaly detection model for identifying anomalous relations in these dynamic log graphs. This model employs a Graph Neural Network (GNN)-based encoder enhanced with transformers to capture content, structural and temporal features. We evaluate our proposed method on three datasets, and the results demonstrate the effectiveness of GLAD in detecting anomalies indicated by varying relational patterns.

Related papers

• Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks [50.42343781348247]
  We develop a graph Poisson factor analysis (GPFA) which provides analytic conditional posteriors to improve the inference accuracy. We also extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.
  arXiv  Detail & Related papers  (2024-10-13T02:22:14Z)
• Log2graphs: An Unsupervised Framework for Log Anomaly Detection with Efficient Feature Extraction [1.474723404975345]
  High cost of manual annotation and dynamic nature of usage scenarios present major challenges to effective log analysis. This study proposes a novel log feature extraction model called DualGCN-LogAE, designed to adapt to various scenarios. We also introduce Log2graphs, an unsupervised log anomaly detection method based on the feature extractor.
  arXiv  Detail & Related papers  (2024-09-18T11:35:58Z)
• BOURNE: Bootstrapped Self-supervised Learning Framework for Unified Graph Anomaly Detection [50.26074811655596]
  We propose a novel unified graph anomaly detection framework based on bootstrapped self-supervised learning (named BOURNE) By swapping the context embeddings between nodes and edges, we enable the mutual detection of node and edge anomalies. BOURNE can eliminate the need for negative sampling, thereby enhancing its efficiency in handling large graphs.
  arXiv  Detail & Related papers  (2023-07-28T00:44:57Z)
• Graph Neural Networks based Log Anomaly Detection and Explanation [19.66344385835598]
  Event logs are widely used to record the status of high-tech systems. Most existing log anomaly detection methods take a log event count matrix or log event sequences as input. We propose a graph-based method for unsupervised log anomaly detection, dubbed Logs2Graphs.
  arXiv  Detail & Related papers  (2023-07-02T09:38:43Z)
• LogGD:Detecting Anomalies from System Logs by Graph Neural Networks [14.813971618949068]
  We propose a novel graph-based log anomaly detection method, LogGD, to effectively address the issue. We exploit the powerful capability of Graph Transformer Neural Network, which combines graph structure and node semantics for log-based anomaly detection.
  arXiv  Detail & Related papers  (2022-09-16T11:51:58Z)
• Topological Data Analysis for Anomaly Detection in Host-Based Logs [1.0878040851638]
  We present an approach that builds a filtration of simplicial complexes directly from Windows logs, enabling analysis of their intrinsic structure using topological tools. We end by discussing the potential for our methods to be used as part of an explainable framework for anomaly detection.
  arXiv  Detail & Related papers  (2022-04-25T20:41:02Z)
• Explicit Pairwise Factorized Graph Neural Network for Semi-Supervised Node Classification [59.06717774425588]
  We propose the Explicit Pairwise Factorized Graph Neural Network (EPFGNN), which models the whole graph as a partially observed Markov Random Field. It contains explicit pairwise factors to model output-output relations and uses a GNN backbone to model input-output relations. We conduct experiments on various datasets, which shows that our model can effectively improve the performance for semi-supervised node classification on graphs.
  arXiv  Detail & Related papers  (2021-07-27T19:47:53Z)
• Log2NS: Enhancing Deep Learning Based Analysis of Logs With Formal to Prevent Survivorship Bias [0.37943450391498496]
  We introduce log to Neuro-symbolic (Log2NS), a framework that combines probabilistic analysis from machine learning (ML) techniques on observational data with certainties derived from symbolic reasoning on an underlying formal model. Log2NS provides an ability to query from static logs and correlation engines for positive instances, as well as formal reasoning for negative and unseen instances.
  arXiv  Detail & Related papers  (2021-05-29T00:01:08Z)
• Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs [59.04636530383049]
  We propose Logsy, a classification-based method to learn log representations. We show an average improvement of 0.25 in the F1 score, compared to the previous methods.
  arXiv  Detail & Related papers  (2020-08-21T07:26:55Z)
• Structural Temporal Graph Neural Networks for Anomaly Detection in Dynamic Graphs [54.13919050090926]
  We propose an end-to-end structural temporal Graph Neural Network model for detecting anomalous edges in dynamic graphs. In particular, we first extract the $h$-hop enclosing subgraph centered on the target edge and propose the node labeling function to identify the role of each node in the subgraph. Based on the extracted features, we utilize Gated recurrent units (GRUs) to capture the temporal information for anomaly detection.
  arXiv  Detail & Related papers  (2020-05-15T09:17:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.