AnomalyGen: An Automated Semantic Log Sequence Generation Framework with LLM for Anomaly Detection

• URL: http://arxiv.org/abs/2504.12250v1
• Date: Wed, 16 Apr 2025 16:54:38 GMT
• Title: AnomalyGen: An Automated Semantic Log Sequence Generation Framework with LLM for Anomaly Detection
• Authors: Xinyu Li, Yingtong Huo, Chenxi Mao, Shiwen Shan, Yuxin Su, Dan Li, Zibin Zheng,
• Abstract summary: AnomalyGen is the first automated log synthesis framework specifically designed for anomaly detection.<n>Our framework integrates enhanced program analysis with Chain-of-Thought reasoning (CoT reasoning) to enable iterative log generation and anomaly annotation.<n>When augmenting benchmark datasets with synthesized logs, we observe maximum F1-score improvements of 3.7%.
• Score: 25.83270938475311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The scarcity of high-quality public log datasets has become a critical bottleneck in advancing log-based anomaly detection techniques. Current datasets exhibit three fundamental limitations: (1) incomplete event coverage, (2) artificial patterns introduced by static analysis-based generation frameworks, and (3) insufficient semantic awareness. To address these challenges, we present AnomalyGen, the first automated log synthesis framework specifically designed for anomaly detection. Our framework introduces a novel four-phase architecture that integrates enhanced program analysis with Chain-of-Thought reasoning (CoT reasoning), enabling iterative log generation and anomaly annotation without requiring physical system execution. Evaluations on Hadoop and HDFS distributed systems demonstrate that AnomalyGen achieves substantially broader log event coverage (38-95 times improvement over existing datasets) while producing more operationally realistic log sequences compared to static analysis-based approaches. When augmenting benchmark datasets with synthesized logs, we observe maximum F1-score improvements of 3.7% (average 1.8% improvement across three state-of-the-art anomaly detection models). This work not only establishes a high-quality benchmarking resource for automated log analysis but also pioneers a new paradigm for applying large language models (LLMs) in software engineering workflows.

Related papers

• Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute [61.00662702026523]
  We propose a unified Test-Time Compute scaling framework that leverages increased inference-time instead of larger models.<n>Our framework incorporates two complementary strategies: internal TTC and external TTC.<n>We demonstrate our textbf32B model achieves a 46% issue resolution rate, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1.
  arXiv  Detail & Related papers  (2025-03-31T07:31:32Z)
• Scaling Laws of Synthetic Data for Language Models [132.67350443447611]
  We introduce SynthLLM, a scalable framework that transforms pre-training corpora into diverse, high-quality synthetic datasets.<n>Our approach achieves this by automatically extracting and recombining high-level concepts across multiple documents using a graph algorithm.
  arXiv  Detail & Related papers  (2025-03-25T11:07:12Z)
• LogSD: Detecting Anomalies from System Logs through Self-supervised Learning and Frequency-based Masking [14.784236273395017]
  We propose LogSD, a novel semi-supervised self-supervised learning approach. We show that LogSD significantly outperforms eight state-of-the-art benchmark methods.
  arXiv  Detail & Related papers  (2024-04-17T12:00:09Z)
• LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection [73.69399219776315]
  We propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters.
  arXiv  Detail & Related papers  (2024-01-09T12:55:21Z)
• GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection [49.9884374409624]
  GLAD is a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs. We introduce GLAD, a Graph-based Log Anomaly Detection framework designed to detect anomalies in system logs.
  arXiv  Detail & Related papers  (2023-09-12T04:21:30Z)
• Log-based Anomaly Detection based on EVT Theory with feedback [31.949892354842525]
  We present an accurate, lightweight, and adaptive log-based anomaly detection framework, referred to as SeaLog. Our method introduces a Trie-based Detection Agent (TDA) that employs a lightweight, dynamically-growing trie structure for real-time anomaly detection. To enhance TDA's accuracy in response to evolving log data, we enable it to receive feedback from experts.
  arXiv  Detail & Related papers  (2023-06-08T08:34:58Z)
• SKTR: Trace Recovery from Stochastically Known Logs [7.882975068446842]
  Developments in machine learning together with the increasing usage of sensor data challenge the reliance on deterministic logs. In this work we formulate the task of generating a deterministic log fromally known logs that is as faithful to reality as possible. An effective trace recovery algorithm would be a powerful aid for maintaining credible process mining tools for uncertain settings.
  arXiv  Detail & Related papers  (2022-06-25T15:29:20Z)
• Hybrid Graph Models for Logic Optimization via Spatio-Temporal Information [15.850413267830522]
  Two major concerns that may impede production-ready ML applications in EDA are accuracy requirements and generalization capability. We propose hybrid graph neural network (GNN) based approaches towards highly accurate quality-of-result (QoR) estimations. Evaluation on 3.3 million data points shows that the testing mean absolute percentage error (MAPE) on designs seen unseen during training are no more than 1.2% and 3.1%.
  arXiv  Detail & Related papers  (2022-01-20T21:12:22Z)
• TransLog: A Unified Transformer-based Framework for Log Anomaly Detection [29.29752871868652]
  Ourmethod is comprised of the pretraining and adapter-based tuning stage. Our simple yet efficient approach, with fewer trainable parameters and lower training costs in the target domain, achieves state-of-the-art performance on three benchmarks.
  arXiv  Detail & Related papers  (2021-12-31T10:46:14Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.