Using Large Language Models for Template Detection from Security Event Logs

• URL: http://arxiv.org/abs/2409.05045v1
• Date: Sun, 8 Sep 2024 10:06:54 GMT
• Title: Using Large Language Models for Template Detection from Security Event Logs
• Authors: Risto Vaarandi, Hayretdin Bahsi,
• Abstract summary: Event log analysis techniques are essential for the timely detection of cyber attacks and for assisting security experts with the analysis of past security incidents. The detection of line patterns or templates from unstructured textual event logs has been identified as an important task of event log analysis. This paper investigates the application of Large Language Models (LLMs) for unsupervised detection of templates from unstructured security event logs.
• Score: 0.9217021281095907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In modern IT systems and computer networks, real-time and offline event log analysis is a crucial part of cyber security monitoring. In particular, event log analysis techniques are essential for the timely detection of cyber attacks and for assisting security experts with the analysis of past security incidents. The detection of line patterns or templates from unstructured textual event logs has been identified as an important task of event log analysis since detected templates represent event types in the event log and prepare the logs for downstream online or offline security monitoring tasks. During the last two decades, a number of template mining algorithms have been proposed. However, many proposed algorithms rely on traditional data mining techniques, and the usage of Large Language Models (LLMs) has received less attention so far. Also, most approaches that harness LLMs are supervised, and unsupervised LLM-based template mining remains an understudied area. The current paper addresses this research gap and investigates the application of LLMs for unsupervised detection of templates from unstructured security event logs.

Related papers

• Training-free Anomaly Event Detection via LLM-guided Symbolic Pattern Discovery [70.75963253876628]
  Anomaly event detection plays a crucial role in various real-world applications. We present a training-free framework that integrates open-set object detection with symbolic regression.
  arXiv  Detail & Related papers  (2025-02-09T10:30:54Z)
• LLM-based event log analysis techniques: A survey [1.6180992915701702]
  Event logs record key information on activities that occur on computing devices. Researchers have developed automated techniques to improve the event log analysis process. This paper aims to survey LLM-based event log analysis techniques.
  arXiv  Detail & Related papers  (2025-02-02T05:28:17Z)
• TPLogAD: Unsupervised Log Anomaly Detection Based on Event Templates and Key Parameters [2.8377404383552043]
  We propose TPLogAD, a universal unsupervised method for analyzing unstructured logs. The itemplate2vec and para2vec included in TPLogAD are two efficient and easy-to-implement semantic representation methods for logs. Our experiments on four public log datasets show that TPLogAD outperforms existing log anomaly detection methods.
  arXiv  Detail & Related papers  (2024-11-22T08:25:21Z)
• LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs [0.0]
  The goal of log-based anomaly detection is to automatically detect system anomalies by analyzing the large number of logs generated in a short period of time. Previous studies have used a log to extract templates from unstructured log data and detect anomalies on the basis of patterns of the template occurrences. We propose LogELECTRA, a new log anomaly detection model that analyzes a single line of log messages more deeply on the basis of self-supervised anomaly detection.
  arXiv  Detail & Related papers  (2024-02-16T01:47:02Z)
• Detecting Anomalous Events in Object-centric Business Processes via Graph Neural Networks [55.583478485027]
  This study proposes a novel framework for anomaly detection in business processes. We first reconstruct the process dependencies of the object-centric event logs as attributed graphs. We then employ a graph convolutional autoencoder architecture to detect anomalous events.
  arXiv  Detail & Related papers  (2024-02-14T14:17:56Z)
• RAPID: Training-free Retrieval-based Log Anomaly Detection with PLM considering Token-level information [7.861095039299132]
  The need for log anomaly detection is growing, especially in real-world applications. Traditional deep learning-based anomaly detection models require dataset-specific training, leading to corresponding delays. We introduce RAPID, a model that capitalizes on the inherent features of log data to enable anomaly detection without training delays.
  arXiv  Detail & Related papers  (2023-11-09T06:11:44Z)
• Leveraging a Probabilistic PCA Model to Understand the Multivariate Statistical Network Monitoring Framework for Network Security Anomaly Detection [64.1680666036655]
  We revisit anomaly detection techniques based on PCA from a probabilistic generative model point of view. We have evaluated the mathematical model using two different datasets.
  arXiv  Detail & Related papers  (2023-02-02T13:41:18Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models [59.04636530383049]
  Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users. We propose a framework for anomaly detection in log data, as a major troubleshooting source of system information.
  arXiv  Detail & Related papers  (2021-02-23T09:17:05Z)
• Detecting the Insider Threat with Long Short Term Memory (LSTM) Neural Networks [0.799536002595393]
  In this study, we use deep learning, and most specifically Long Short Term Memory (LSTM) recurrent networks for enabling the detection of insider threats. We demonstrate through a very large, anonymized dataset how LSTM uses the sequenced nature of the data for reducing the search space and making the work of a security analyst more effective.
  arXiv  Detail & Related papers  (2020-07-20T23:29:01Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.