Linnaeus: A highly reusable and adaptable ML based log classification pipeline

• URL: http://arxiv.org/abs/2103.06927v1
• Date: Thu, 11 Mar 2021 19:58:53 GMT
• Title: Linnaeus: A highly reusable and adaptable ML based log classification pipeline
• Abstract summary: In recent times there has been more emphasis on statistical and machine learning (ML) based methods for analyzing logs. In this paper we demonstrate our end-to-end log classification pipeline, Linnaeus. We also demonstrate our solutions for adaptability and re-use, integration towards large scale software development processes, and how we cope with lack of labelled data.
• Score: 2.2099003320482393
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Logs are a common way to record detailed run-time information in software. As modern software systems evolve in scale and complexity, logs have become indispensable to understanding the internal states of the system. At the same time however, manually inspecting logs has become impractical. In recent times, there has been more emphasis on statistical and machine learning (ML) based methods for analyzing logs. While the results have shown promise, most of the literature focuses on algorithms and state-of-the-art (SOTA), while largely ignoring the practical aspects. In this paper we demonstrate our end-to-end log classification pipeline, Linnaeus. Besides showing the more traditional ML flow, we also demonstrate our solutions for adaptability and re-use, integration towards large scale software development processes, and how we cope with lack of labelled data. We hope Linnaeus can serve as a blueprint for, and inspire the integration of, various ML based solutions in other large scale industrial settings.

Related papers

• MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI [92.82821272460423]
  MLS-Bench is a benchmark for evaluating whether AI systems can invent generalizable and scalable ML methods.<n>We study the effects of test-time scaling, adaptive compute allocation, and context provision on agents' discovery performance.
  arXiv  Detail & Related papers  (2026-05-09T04:29:46Z)
• Empirical Characterization of Logging Smells in Machine Learning Code [7.520925824033758]
  We present an empirical study of logging smells in machine learning (ML) projects.<n>We identify 12 categories of logging smells spanning security, metric management, configuration, verbosity, and context-related issues.<n>Our results show that logging smells are widespread in ML systems and vary in frequency and manifestation across projects.
  arXiv  Detail & Related papers  (2026-03-24T23:01:40Z)
• Empirical Characterization of Logging Smells in Machine Learning Code [7.520925824033758]
  This study aims to empirically identify and characterize logging smells in machine learning systems.<n>We propose to conduct a large-scale mining of open-source ML repositories hosted on GitHub to catalogue recurring logging smells.
  arXiv  Detail & Related papers  (2026-01-09T05:48:42Z)
• Generality Is Not Enough: Zero-Label Cross-System Log-Based Anomaly Detection via Knowledge-Level Collaboration [10.873294740040912]
  GeneralLog is a novel collaborative method for zero-label cross-system log anomaly detection.<n>GeneralLog achieves over 90% F1-score under a fully zero-label setting, significantly outperforming existing methods.
  arXiv  Detail & Related papers  (2025-11-08T06:47:28Z)
• R-Log: Incentivizing Log Analysis Capability in LLMs via Reasoning-based Reinforcement Learning [19.713020881817588]
  R-Log is a novel reasoning-based paradigm that mirrors the structured, step-by-step analytical process of human engineers.<n>R-Log is first cold-started on a curated dataset of 2k+ reasoning trajectories, guided by 13 strategies from manual O&M practices.<n> Empirical evaluations on real-world logs show that R-Log outperforms existing methods across five log analysis tasks.
  arXiv  Detail & Related papers  (2025-09-30T09:19:31Z)
• SoK: LLM-based Log Parsing [2.2779174914142346]
  This paper systematically reviews 29 large language models (LLMs)-based log parsing methods. We analyze the learning and prompt-engineering paradigms employed, efficiency- and effectiveness-enhancing techniques, and the role of LLMs in the parsing process.
  arXiv  Detail & Related papers  (2025-04-07T09:41:04Z)
• Virgo: A Preliminary Exploration on Reproducing o1-like MLLM [89.50691075011429]
  Slow-thinking reasoning systems have garnered widespread attention by scaling the thinking time during inference. There is also growing interest in adapting this capability to multimodal large language models (MLLMs) In this paper, we explore a straightforward approach by fine-tuning a capable MLLM with a small amount of textual long-form thought data. We find that these long-form reasoning processes, expressed in natural language, can be effectively transferred to MLLMs.
  arXiv  Detail & Related papers  (2025-01-03T17:14:16Z)
• Studying and Benchmarking Large Language Models For Log Level Suggestion [49.176736212364496]
  Large Language Models (LLMs) have become a focal point of research across various domains. This paper investigates the impact of characteristics and learning paradigms on the performance of 12 open-source LLMs in log level suggestion.
  arXiv  Detail & Related papers  (2024-10-11T03:52:17Z)
• LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models [19.657278472819588]
  We introduce Log-LLM, a novel log integrated with LLM capabilities. We address the intricate challenge of parsing granularity, proposing a new metric to allow users to calibrate granularity to their specific needs. Our method's efficacy is empirically demonstrated through evaluations on the Loghub-2k and the large-scale LogPub benchmark.
  arXiv  Detail & Related papers  (2024-08-25T05:34:24Z)
• DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph [70.79413606968814]
  We introduce Dynamic Evaluation of LLMs via Adaptive Reasoning Graph Evolvement (DARG) to dynamically extend current benchmarks with controlled complexity and diversity. Specifically, we first extract the reasoning graphs of data points in current benchmarks and then perturb the reasoning graphs to generate novel testing data. Such newly generated test samples can have different levels of complexity while maintaining linguistic diversity similar to the original benchmarks.
  arXiv  Detail & Related papers  (2024-06-25T04:27:53Z)
• LUNAR: Unsupervised LLM-based Log Parsing [34.344687402936835]
  We propose LUNAR, an unsupervised-based method for efficient and off-the-shelf log parsing. Our key insight is that while LLMs may struggle with direct log parsing, their performance can be significantly enhanced through comparative analysis. Experiments on large-scale public datasets demonstrate that LUNAR significantly outperforms state-of-the-art log crafts in terms of accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-06-11T11:32:01Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging [33.522495018321386]
  We introduce a cutting-edge textbfLog parsing framework with textbfEntropy sampling and Chain-of-Thought textbfMerging (Lemur) We propose a novel sampling method inspired by information entropy, which efficiently clusters typical logs. Lemur achieves the state-of-the-art performance and impressive efficiency.
  arXiv  Detail & Related papers  (2024-02-28T09:51:55Z)
• Log Parsing Evaluation in the Era of Modern Software Systems [47.370291246632114]
  We focus on one integral part of automated log analysis, log parsing, which is the prerequisite to deriving any insights from logs. Our investigation reveals problematic aspects within the log parsing field, particularly its inefficiency in handling heterogeneous real-world logs. We propose a tool, Logchimera, that enables estimating log parsing performance in industry contexts.
  arXiv  Detail & Related papers  (2023-08-17T14:19:22Z)
• On the Effectiveness of Log Representation for Log-based Anomaly Detection [12.980238412281471]
  This work investigates and compares the commonly adopted log representation techniques from previous log analysis research. We select six log representation techniques and evaluate them with seven ML models and four public log datasets. We also examine the impacts of the log parsing process and the different feature aggregation approaches when they are employed with log representation techniques.
  arXiv  Detail & Related papers  (2023-08-17T02:18:59Z)
• LogAI: A Library for Log Analytics and Intelligence [27.889928073709516]
  LogAI is a one-stop open source library for log analytics and intelligence. It supports tasks such as log summarization, log clustering and log anomaly detection. LogAI provides a unified model interface and provides popular time-series, statistical learning and deep learning models.
  arXiv  Detail & Related papers  (2023-01-31T05:08:39Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.