A Novel GPT-Based Framework for Anomaly Detection in System Logs

• URL: http://arxiv.org/abs/2510.16044v1
• Date: Thu, 16 Oct 2025 15:17:39 GMT
• Title: A Novel GPT-Based Framework for Anomaly Detection in System Logs
• Authors: Zeng Zhang, Wenjie Yin, Xiaoqi Li,
• Abstract summary: This paper proposes an intelligent detection method for system logs based on Genera- tive Pre-trained Transformers (GPT)<n>The efficacy of this approach is attributable to a combination of structured input and a Focal Loss op timization strategy.<n>The GPT-2 model significantly outperforms the unoptimized model in a range of key metrics, including precision, recall, and F1 score.
• Score: 4.92711268765052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identification of anomalous events within system logs constitutes a pivotal element within the frame- work of cybersecurity defense strategies. However, this process faces numerous challenges, including the management of substantial data volumes, the distribution of anomalies, and the precision of con- ventional methods. To address this issue, the present paper puts forward a proposal for an intelligent detection method for system logs based on Genera- tive Pre-trained Transformers (GPT). The efficacy of this approach is attributable to a combination of structured input design and a Focal Loss op- timization strategy, which collectively result in a substantial enhancement of the performance of log anomaly detection. The initial approach involves the conversion of raw logs into event ID sequences through the use of the Drain parser. Subsequently, the Focal Loss loss function is employed to address the issue of class imbalance. The experimental re- sults demonstrate that the optimized GPT-2 model significantly outperforms the unoptimized model in a range of key metrics, including precision, recall, and F1 score. In specific tasks, comparable or superior performance has been demonstrated to that of the GPT-3.5 API.

Related papers

• Registration is a Powerful Rotation-Invariance Learner for 3D Anomaly Detection [64.0168648353038]
  3D anomaly detection in point-cloud data is critical for industrial quality control, aiming to identify structural defects with high reliability.<n>Current memory bank-based methods often suffer from inconsistent feature transformations and limited discriminative capacity.<n>We propose a registration-induced, rotation-invariant feature extraction framework that integrates the objectives of point-cloud registration and memory-based anomaly detection.
  arXiv  Detail & Related papers  (2025-10-19T14:56:38Z)
• Source-Free Object Detection with Detection Transformer [59.33653163035064]
  Source-Free Object Detection (SFOD) enables knowledge transfer from a source domain to an unsupervised target domain for object detection without access to source data.<n>Most existing SFOD approaches are either confined to conventional object detection (OD) models like Faster R-CNN or designed as general solutions without tailored adaptations for novel OD architectures, especially Detection Transformer (DETR)<n>In this paper, we introduce Feature Reweighting ANd Contrastive Learning NetworK (FRANCK), a novel SFOD framework specifically designed to perform query-centric feature enhancement for DETRs.
  arXiv  Detail & Related papers  (2025-10-13T07:35:04Z)
• FastRef:Fast Prototype Refinement for Few-Shot Industrial Anomaly Detection [18.487111110151115]
  Few-shot industrial anomaly detection (FS-IAD) presents a critical challenge for practical automated inspection systems.<n>We propose FastRef, a novel and efficient prototype refinement framework for FS-IAD.<n>For comprehensive evaluation, we integrate FastRef with three competitive prototype-based FS-IAD methods: PatchCore, FastRecon, WinCLIP, and AnomalyDINO.
  arXiv  Detail & Related papers  (2025-06-26T15:46:28Z)
• Distributed Log-driven Anomaly Detection System based on Evolving Decision Making [4.183506125389502]
  CEDLog is a framework that implements distributed computing for scalable processing by integrating Apache Airflow and Dask.<n>In CEDLog, anomalies are detected through the synthesis of Multi-layer Perceptron (MLP) and Graph Convolutional Networks (GCNs) using critical features present in event logs.
  arXiv  Detail & Related papers  (2025-04-03T06:50:30Z)
• OMLog: Online Log Anomaly Detection for Evolving System with Meta-learning [10.181157278476428]
  OMLog is a real-time and reliable online log anomaly detection model. We introduce a maximum mean discrepancy-based distribution shift detection method. We also design an online learning mechanism based on meta-learning, which can effectively learn the highly repetitive patterns of log sequences.
  arXiv  Detail & Related papers  (2024-10-22T01:50:07Z)
• PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection [65.24854366973794]
  Node-level graph anomaly detection (GAD) plays a critical role in identifying anomalous nodes from graph-structured data in domains such as medicine, social networks, and e-commerce. We introduce a simple method termed PREprocessing and Matching (PREM for short) to improve the efficiency of GAD. Our approach streamlines GAD, reducing time and memory consumption while maintaining powerful anomaly detection capabilities.
  arXiv  Detail & Related papers  (2023-10-18T02:59:57Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• A hybrid feature learning approach based on convolutional kernels for ATM fault prediction using event-log data [5.859431341476405]
  We present a predictive model based on a convolutional kernel (MiniROCKET and HYDRA) to extract features from event-log data. The proposed methodology is applied to a significant real-world collected dataset. The model was integrated into a container-based decision support system to support operators in the timely maintenance of ATMs.
  arXiv  Detail & Related papers  (2023-05-17T08:55:53Z)
• BERT-based Chinese Text Classification for Emergency Domain with a Novel Loss Function [9.028459232146474]
  This paper proposes an automatic Chinese text categorization method for solving the emergency event report classification problem. To overcome the data imbalance problem in the distribution of emergency event categories, a novel loss function is proposed to improve the performance of the BERT-based model. The proposed method has achieved the best performance in terms of accuracy, weighted-precision, weighted-recall, and weighted-F1 values.
  arXiv  Detail & Related papers  (2021-04-09T05:25:00Z)
• Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs [59.04636530383049]
  We propose Logsy, a classification-based method to learn log representations. We show an average improvement of 0.25 in the F1 score, compared to the previous methods.
  arXiv  Detail & Related papers  (2020-08-21T07:26:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.