USTEP: Structuration des logs en flux gr{\^a}ce {\`a} un arbre de recherche {\'e}volutif

• URL: http://arxiv.org/abs/2304.12331v1
• Date: Mon, 24 Apr 2023 09:12:00 GMT
• Title: USTEP: Structuration des logs en flux gr{\^a}ce {\`a} un arbre de recherche {\'e}volutif
• Authors: Arthur Vervaet (ISEP), Raja Chiky (ISEP), Mar Callau-Zori
• Abstract summary: Parsing log messages to structure their format is a classic preliminary step for log-mining tasks. We propose USTEP, an online log parsing method based on an evolving tree structure.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Logs record valuable system information at runtime. They are widely used by data-driven approaches for development and monitoring purposes. Parsing log messages to structure their format is a classic preliminary step for log-mining tasks. As they appear upstream, parsing operations can become a processing time bottleneck for downstream applications. The quality of parsing also has a direct influence on their efficiency. Here, we propose USTEP, an online log parsing method based on an evolving tree structure. Evaluation results on a wide panel of datasets coming from different real-world systems demonstrate USTEP superiority in terms of both effectiveness and robustness when compared to other online methods.

Related papers

• HELP: Hierarchical Embeddings-based Log Parsing [0.25112747242081457]
  Logs are a first-hand source of information for software maintenance and failure diagnosis. Log parsing is a prerequisite for automated log analysis tasks such as anomaly detection, troubleshooting, and root cause analysis. Existing online parsing algorithms are susceptible to log drift, where slight log changes create false positives that drown out real anomalies.
  arXiv  Detail & Related papers  (2024-08-15T17:54:31Z)
• LUNAR: Unsupervised LLM-based Log Parsing [34.344687402936835]
  We propose LUNAR, an unsupervised-based method for efficient and off-the-shelf log parsing. Our key insight is that while LLMs may struggle with direct log parsing, their performance can be significantly enhanced through comparative analysis. Experiments on large-scale public datasets demonstrate that LUNAR significantly outperforms state-of-the-art log crafts in terms of accuracy and efficiency.
  arXiv  Detail & Related papers  (2024-06-11T11:32:01Z)
• Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging [36.39653543676875]
  We introduce a cutting-edge textbfLog parsing framework with textbfEntropy sampling and Chain-of-Thought textbfMerging (Lemur) We propose a novel sampling method inspired by information entropy, which efficiently clusters typical logs. Lemur achieves the state-of-the-art performance and impressive efficiency.
  arXiv  Detail & Related papers  (2024-02-28T09:51:55Z)
• LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection [73.69399219776315]
  We propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters.
  arXiv  Detail & Related papers  (2024-01-09T12:55:21Z)
• A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We? [42.56249610409624]
  We provide a new collection of annotated log datasets, denoted Loghub-2.0, which can better reflect the characteristics of log data in real-world software systems. We conduct a thorough re-evaluation of 15 state-of-the-art logs in a more rigorous and practical setting. Particularly, we introduce a new evaluation metric to mitigate the sensitivity of existing metrics to imbalanced data distributions.
  arXiv  Detail & Related papers  (2023-08-21T16:24:15Z)
• Log Parsing Evaluation in the Era of Modern Software Systems [47.370291246632114]
  We focus on one integral part of automated log analysis, log parsing, which is the prerequisite to deriving any insights from logs. Our investigation reveals problematic aspects within the log parsing field, particularly its inefficiency in handling heterogeneous real-world logs. We propose a tool, Logchimera, that enables estimating log parsing performance in industry contexts.
  arXiv  Detail & Related papers  (2023-08-17T14:19:22Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs [59.04636530383049]
  We propose Logsy, a classification-based method to learn log representations. We show an average improvement of 0.25 in the F1 score, compared to the previous methods.
  arXiv  Detail & Related papers  (2020-08-21T07:26:55Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.