LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model

• URL: http://arxiv.org/abs/2111.09564v3
• Date: Sun, 23 Jul 2023 16:02:01 GMT
• Title: LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model
• Authors: Yukyung Lee, Jina Kim and Pilsung Kang
• Abstract summary: The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template. In this study, we propose LAnoBERT, exhibiting excellent natural language processing performance.
• Score: 12.00171674362062
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The system log generated in a computer system refers to large-scale data that are collected simultaneously and used as the basic data for determining errors, intrusion and abnormal behaviors. The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention, which is a critical problem in the industry. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template using a parser. Particularly, a template corresponding to a specific event should be defined in advance for all the log data using which the information within the log key may get lost. In this study, we propose LAnoBERT, a parser free system log anomaly detection method that uses the BERT model, exhibiting excellent natural language processing performance. The proposed method, LAnoBERT, learns the model through masked language modeling, which is a BERT-based pre-training method, and proceeds with unsupervised learning-based anomaly detection using the masked language modeling loss function per log key during the test process. In addition, we also propose an efficient inference process to establish a practically applicable pipeline to the actual system. Experiments on three well-known log datasets, i.e., HDFS, BGL, and Thunderbird, show that not only did LAnoBERT yield a higher anomaly detection performance compared to unsupervised learning-based benchmark models, but also it resulted in a comparable performance with supervised learning-based benchmark models.

Related papers

• LogLLM: Log-based Anomaly Detection Using Large Language Models [8.03646578793411]
  We propose LogLLM, a log-based anomaly detection framework that leverages large language models (LLMs) LogLLM employs BERT for extracting semantic vectors from log messages, while utilizing Llama, a transformer decoder-based model, for classifying log sequences. Our framework is trained through a novel three-stage procedure designed to enhance performance and adaptability.
  arXiv  Detail & Related papers  (2024-11-13T12:18:00Z)
• LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs [0.0]
  The goal of log-based anomaly detection is to automatically detect system anomalies by analyzing the large number of logs generated in a short period of time. Previous studies have used a log to extract templates from unstructured log data and detect anomalies on the basis of patterns of the template occurrences. We propose LogELECTRA, a new log anomaly detection model that analyzes a single line of log messages more deeply on the basis of self-supervised anomaly detection.
  arXiv  Detail & Related papers  (2024-02-16T01:47:02Z)
• RAPID: Training-free Retrieval-based Log Anomaly Detection with PLM considering Token-level information [7.861095039299132]
  The need for log anomaly detection is growing, especially in real-world applications. Traditional deep learning-based anomaly detection models require dataset-specific training, leading to corresponding delays. We introduce RAPID, a model that capitalizes on the inherent features of log data to enable anomaly detection without training delays.
  arXiv  Detail & Related papers  (2023-11-09T06:11:44Z)
• LogGPT: Log Anomaly Detection via GPT [15.790373280124196]
  We propose LogGPT, a novel framework that employs GPT for log anomaly detection. LogGPT is first trained to predict the next log entry based on the preceding sequence. We propose a novel reinforcement learning strategy to finetune the model specifically for the log anomaly detection task.
  arXiv  Detail & Related papers  (2023-09-25T19:29:50Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Log-based Anomaly Detection Without Log Parsing [7.66638994053231]
  We propose NeuralLog, a novel log-based anomaly detection approach that does not require log parsing. Our experimental results show that the proposed approach can effectively understand the semantic meaning of log messages. Overall, NeuralLog achieves F1-scores greater than 0.95 on four public datasets, outperforming the existing approaches.
  arXiv  Detail & Related papers  (2021-08-04T10:42:13Z)
• Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models [59.04636530383049]
  Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users. We propose a framework for anomaly detection in log data, as a major troubleshooting source of system information.
  arXiv  Detail & Related papers  (2021-02-23T09:17:05Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs [59.04636530383049]
  We propose Logsy, a classification-based method to learn log representations. We show an average improvement of 0.25 in the F1 score, compared to the previous methods.
  arXiv  Detail & Related papers  (2020-08-21T07:26:55Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.