ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model

• URL: http://arxiv.org/abs/2505.13496v1
• Date: Thu, 15 May 2025 17:31:40 GMT
• Title: ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model
• Authors: Przemek Pospieszny, Wojciech Mormul, Karolina Szyndler, Sanjeev Kumar,
• Abstract summary: ADALog is an adaptive, unsupervised anomaly detection framework.<n>It operates on individual unstructured logs, extracts intra-log contextual relationships, and performs adaptive thresholding on normal data.<n>We evaluate ADALog on benchmark datasets BGL, Thunderbird, and Spirit.
• Score: 2.55347686868565
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Modern software systems generate extensive heterogeneous log data with dynamic formats, fragmented event sequences, and varying temporal patterns, making anomaly detection both crucial and challenging. To address these complexities, we propose ADALog, an adaptive, unsupervised anomaly detection framework designed for practical applicability across diverse real-world environments. Unlike traditional methods reliant on log parsing, strict sequence dependencies, or labeled data, ADALog operates on individual unstructured logs, extracts intra-log contextual relationships, and performs adaptive thresholding on normal data. The proposed approach utilizes a transformer-based, pretrained bidirectional encoder with a masked language modeling task, fine-tuned on normal logs to capture domain-specific syntactic and semantic patterns essential for accurate anomaly detection. Anomalies are identified via token-level reconstruction probabilities, aggregated into log-level scores, with adaptive percentile-based thresholding calibrated only on normal data. This allows the model to dynamically adapt to evolving system behaviors while avoiding rigid, heuristic-based thresholds common in traditional systems. We evaluate ADALog on benchmark datasets BGL, Thunderbird, and Spirit, showing strong generalization and competitive performance compared to state-of-the-art supervised and unsupervised methods. Additional ablation studies examine the effects of masking, fine-tuning, and token positioning on model behavior and interpretability.

Related papers

• OMLog: Online Log Anomaly Detection for Evolving System with Meta-learning [10.181157278476428]
  OMLog is a real-time and reliable online log anomaly detection model. We introduce a maximum mean discrepancy-based distribution shift detection method. We also design an online learning mechanism based on meta-learning, which can effectively learn the highly repetitive patterns of log sequences.
  arXiv  Detail & Related papers  (2024-10-22T01:50:07Z)
• Trajectory Anomaly Detection with Language Models [21.401931052512595]
  This paper presents a novel approach for trajectory anomaly detection using an autoregressive causal-attention model, termed LM-TAD. By treating trajectories as sequences of tokens, our model learns the probability distributions over trajectories, enabling the identification of anomalous locations with high precision. Our experiments demonstrate the effectiveness of LM-TAD on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-09-18T17:33:31Z)
• Pattern-Based Time-Series Risk Scoring for Anomaly Detection and Alert Filtering -- A Predictive Maintenance Case Study [3.508168174653255]
  We propose a fast and efficient approach to anomaly detection and alert filtering based on sequential pattern similarities. We show how this approach can be leveraged for a variety of purposes involving anomaly detection on a large scale real-world industrial system.
  arXiv  Detail & Related papers  (2024-05-24T20:27:45Z)
• Detecting Anomalies in Dynamic Graphs via Memory enhanced Normality [39.476378833827184]
  Anomaly detection in dynamic graphs presents a significant challenge due to the temporal evolution of graph structures and attributes. We introduce a novel spatial- temporal memories-enhanced graph autoencoder (STRIPE) STRIPE significantly outperforms existing methods with 5.8% improvement in AUC scores and 4.62X faster in training time.
  arXiv  Detail & Related papers  (2024-03-14T02:26:10Z)
• MLAD: A Unified Model for Multi-system Log Anomaly Detection [35.68387377240593]
  We propose MLAD, a novel anomaly detection model that incorporates semantic relational reasoning across multiple systems. Specifically, we employ Sentence-bert to capture the similarities between log sequences and convert them into highly-dimensional learnable semantic vectors. We revamp the formulas of the Attention layer to discern the significance of each keyword in the sequence and model the overall distribution of the multi-system dataset.
  arXiv  Detail & Related papers  (2024-01-15T12:51:13Z)
• GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection [49.9884374409624]
  GLAD is a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs. We introduce GLAD, a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs.
  arXiv  Detail & Related papers  (2023-09-12T04:21:30Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Adaptive Memory Networks with Self-supervised Learning for Unsupervised Anomaly Detection [54.76993389109327]
  Unsupervised anomaly detection aims to build models to detect unseen anomalies by only training on the normal data. We propose a novel approach called Adaptive Memory Network with Self-supervised Learning (AMSL) to address these challenges. AMSL incorporates a self-supervised learning module to learn general normal patterns and an adaptive memory fusion module to learn rich feature representations.
  arXiv  Detail & Related papers  (2022-01-03T03:40:21Z)
• LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model [12.00171674362062]
  The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template. In this study, we propose LAnoBERT, exhibiting excellent natural language processing performance.
  arXiv  Detail & Related papers  (2021-11-18T07:46:35Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs [59.04636530383049]
  We propose Logsy, a classification-based method to learn log representations. We show an average improvement of 0.25 in the F1 score, compared to the previous methods.
  arXiv  Detail & Related papers  (2020-08-21T07:26:55Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.