ContraLog: Log File Anomaly Detection with Contrastive Learning and Masked Language Modeling

• URL: http://arxiv.org/abs/2602.03678v1
• Date: Tue, 03 Feb 2026 15:59:40 GMT
• Title: ContraLog: Log File Anomaly Detection with Contrastive Learning and Masked Language Modeling
• Abstract summary: ContraLog is a log anomaly detection method that predicts continuous message embeddings rather than discrete template IDs.<n>We show that message embeddings generated by ContraLog carry meaningful information and are predictive of anomalies even without sequence context.<n>Results highlight embedding-level prediction as an approach for log anomaly detection, with potential applicability to other event sequences.
• Score: 5.34230216484876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Log files record computational events that reflect system state and behavior, making them a primary source of operational insights in modern computer systems. Automated anomaly detection on logs is therefore critical, yet most established methods rely on log parsers that collapse messages into discrete templates, discarding variable values and semantic content. We propose ContraLog, a parser-free and self-supervised method that reframes log anomaly detection as predicting continuous message embeddings rather than discrete template IDs. ContraLog combines a message encoder that produces rich embeddings for individual log messages with a sequence encoder to model temporal dependencies within sequences. The model is trained with a combination of masked language modeling and contrastive learning to predict masked message embeddings based on the surrounding context. Experiments on the HDFS, BGL, and Thunderbird benchmark datasets empirically demonstrate effectiveness on complex datasets with diverse log messages. Additionally, we find that message embeddings generated by ContraLog carry meaningful information and are predictive of anomalies even without sequence context. These results highlight embedding-level prediction as an approach for log anomaly detection, with potential applicability to other event sequences.

Related papers

• An End-to-End Framework for Functionality-Embedded Provenance Graph Construction and Threat Interpretation [2.0740117800511215]
  Auto-Prov is an end-to-end framework for provenance graph-based anomaly detectors.<n>It clusters unseen log types and efficiently extracts provenance edges and entity-level information.<n>It infers system-level functional context for both known and previously unseen system entities.<n>It produces stable, interpretable attack summaries that remain robust under system evolution.
  arXiv  Detail & Related papers  (2026-03-17T19:40:05Z)
• ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model [2.55347686868565]
  ADALog is an adaptive, unsupervised anomaly detection framework.<n>It operates on individual unstructured logs, extracts intra-log contextual relationships, and performs adaptive thresholding on normal data.<n>We evaluate ADALog on benchmark datasets BGL, Thunderbird, and Spirit.
  arXiv  Detail & Related papers  (2025-05-15T17:31:40Z)
• LogLLM: Log-based Anomaly Detection Using Large Language Models [7.7704116297749675]
  We propose LogLLM, a log-based anomaly detection framework that leverages large language models (LLMs)<n>LogLLM employs BERT for extracting semantic vectors from log messages, while utilizing Llama, a transformer decoder-based model, for classifying log sequences.<n>Our framework is trained through a novel three-stage procedure designed to enhance performance and adaptability.
  arXiv  Detail & Related papers  (2024-11-13T12:18:00Z)
• DALD: Improving Logits-based Detector without Logits from Black-box LLMs [56.234109491884126]
  Large Language Models (LLMs) have revolutionized text generation, producing outputs that closely mimic human writing. We present Distribution-Aligned LLMs Detection (DALD), an innovative framework that redefines the state-of-the-art performance in black-box text detection. DALD is designed to align the surrogate model's distribution with that of unknown target LLMs, ensuring enhanced detection capability and resilience against rapid model iterations.
  arXiv  Detail & Related papers  (2024-06-07T19:38:05Z)
• Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging [18.823038918091207]
  We introduce a cutting-edge texttextbfLog parsing framework with textbfEntropy sampling and chain-of-thought textbfMerging (model)<n>To discard the tedious manual rules, we propose a novel sampling method inspired by information entropy, which efficiently clusters typical logs.<n>We have conducted experiments on large-scale public datasets.
  arXiv  Detail & Related papers  (2024-02-28T09:51:55Z)
• GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection [49.9884374409624]
  GLAD is a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs. We introduce GLAD, a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs.
  arXiv  Detail & Related papers  (2023-09-12T04:21:30Z)
• LogLG: Weakly Supervised Log Anomaly Detection via Log-Event Graph Construction [31.31712326361932]
  We propose a novel weakly supervised log anomaly detection framework, named LogLG, to explore the semantic connections among keywords from sequences. Specifically, we design an end-to-end iterative process, where the keywords of unlabeled logs are first extracted to construct a log-event graph. Then, we build a subgraph annotator to generate pseudo labels for unlabeled log sequences.
  arXiv  Detail & Related papers  (2022-08-23T09:32:19Z)
• LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model [12.00171674362062]
  The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template. In this study, we propose LAnoBERT, exhibiting excellent natural language processing performance.
  arXiv  Detail & Related papers  (2021-11-18T07:46:35Z)
• Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models [59.04636530383049]
  Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users. We propose a framework for anomaly detection in log data, as a major troubleshooting source of system information.
  arXiv  Detail & Related papers  (2021-02-23T09:17:05Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder [104.25716317141321]
  We propose an approach that automatically finds evidence for an event from a large text corpus, and leverages the evidence to guide the generation of inferential texts. Our approach provides state-of-the-art performance on both Event2Mind and ATOMIC datasets.
  arXiv  Detail & Related papers  (2020-06-15T02:59:52Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.