LogTinyLLM: Tiny Large Language Models Based Contextual Log Anomaly Detection

• URL: http://arxiv.org/abs/2507.11071v1
• Date: Tue, 15 Jul 2025 08:04:31 GMT
• Title: LogTinyLLM: Tiny Large Language Models Based Contextual Log Anomaly Detection
• Authors: Isaiah Thompson Ocansey, Ritwik Bhattacharya, Tanmay Sen,
• Abstract summary: This paper proposes parameter efficient finetuning specifically low rank adaptation (LoRA) and adapter based approaches for finding contextual anomalies in sequence of logs in large log data set.<n>The results show that LoRA based finetuning provides substantial performance improvements of 18 to 19 percentage over LogBert based full finetuning approach.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Log anomaly detection using traditional rule based or deep learning based methods is often challenging due to the large volume and highly complex nature of log sequence. So effective way of detection of anomalous sequence of logs is crucial for system maintenance and development. This paper proposes parameter efficient finetuning specifically low rank adaptation (LoRA) and adapter based approaches for finding contextual anomalies in sequence of logs in large log data set. It compares different tiny large language models (LLMs) on the Thunderbird dataset. The results show that LoRA based finetuning provides substantial performance improvements of 18 to 19 percentage over LogBert based full finetuning approach, achieving accuracy scores between 97.76% and 98.83% compared to 79.37%.

Related papers

• LogPurge: Log Data Purification for Anomaly Detection via Rule-Enhanced Filtering [16.01074159812065]
  We propose a rule-enhanced purification framework, LogPurge, that automatically selects a sufficient subset of normal log sequences to train a anomaly detection model.<n>Our experiments, conducted on two public datasets and one industrial dataset, show that our method significantly removes an average of 98.74% of anomalies while retaining 82.39% of normal samples.
  arXiv  Detail & Related papers  (2025-11-18T02:41:18Z)
• Accelerated Test-Time Scaling with Model-Free Speculative Sampling [58.69141724095398]
  We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach.<n>We show that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding.<n>As a model-free approach, STAND can be applied to any existing language model without additional training.
  arXiv  Detail & Related papers  (2025-06-05T07:31:18Z)
• LogLLM: Log-based Anomaly Detection Using Large Language Models [7.7704116297749675]
  We propose LogLLM, a log-based anomaly detection framework that leverages large language models (LLMs)<n>LogLLM employs BERT for extracting semantic vectors from log messages, while utilizing Llama, a transformer decoder-based model, for classifying log sequences.<n>Our framework is trained through a novel three-stage procedure designed to enhance performance and adaptability.
  arXiv  Detail & Related papers  (2024-11-13T12:18:00Z)
• LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs [0.0]
  The goal of log-based anomaly detection is to automatically detect system anomalies by analyzing the large number of logs generated in a short period of time. Previous studies have used a log to extract templates from unstructured log data and detect anomalies on the basis of patterns of the template occurrences. We propose LogELECTRA, a new log anomaly detection model that analyzes a single line of log messages more deeply on the basis of self-supervised anomaly detection.
  arXiv  Detail & Related papers  (2024-02-16T01:47:02Z)
• LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection [73.69399219776315]
  We propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters.
  arXiv  Detail & Related papers  (2024-01-09T12:55:21Z)
• LogGPT: Log Anomaly Detection via GPT [15.790373280124196]
  We propose LogGPT, a novel framework that employs GPT for log anomaly detection. LogGPT is first trained to predict the next log entry based on the preceding sequence. We propose a novel reinforcement learning strategy to finetune the model specifically for the log anomaly detection task.
  arXiv  Detail & Related papers  (2023-09-25T19:29:50Z)
• PULL: Reactive Log Anomaly Detection Based On Iterative PU Learning [58.85063149619348]
  We propose PULL, an iterative log analysis method for reactive anomaly detection based on estimated failure time windows. Our evaluation shows that PULL consistently outperforms ten benchmark baselines across three different datasets.
  arXiv  Detail & Related papers  (2023-01-25T16:34:43Z)
• LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model [12.00171674362062]
  The aim of system log anomaly detection is to promptly identify anomalies while minimizing human intervention. Previous studies performed anomaly detection through algorithms after converting various forms of log data into a standardized template. In this study, we propose LAnoBERT, exhibiting excellent natural language processing performance.
  arXiv  Detail & Related papers  (2021-11-18T07:46:35Z)
• Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models [59.04636530383049]
  Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users. We propose a framework for anomaly detection in log data, as a major troubleshooting source of system information.
  arXiv  Detail & Related papers  (2021-02-23T09:17:05Z)
• Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs [59.04636530383049]
  We propose Logsy, a classification-based method to learn log representations. We show an average improvement of 0.25 in the F1 score, compared to the previous methods.
  arXiv  Detail & Related papers  (2020-08-21T07:26:55Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.