Learning Representations on Logs for AIOps
- URL: http://arxiv.org/abs/2308.11526v1
- Date: Fri, 18 Aug 2023 20:34:46 GMT
- Title: Learning Representations on Logs for AIOps
- Authors: Pranjal Gupta and Harshit Kumar and Debanjana Kar and Karan Bhukar and
Pooja Aggarwal and Prateeti Mohapatra
- Abstract summary: Large Language Models (LLMs) are trained using self-supervision on a vast amount of unlabeled data.
This paper introduces a LLM for log data which is trained on public and proprietary log data.
Our proposed LLM, trained on public and proprietary log data, offers superior performance on multiple downstream tasks.
- Score: 6.47086647390439
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: AI for IT Operations (AIOps) is a powerful platform that Site Reliability
Engineers (SREs) use to automate and streamline operational workflows with
minimal human intervention. Automated log analysis is a critical task in AIOps
as it provides key insights for SREs to identify and address ongoing faults.
Tasks such as log format detection, log classification, and log parsing are key
components of automated log analysis. Most of these tasks require supervised
learning; however, there are multiple challenges due to limited labelled log
data and the diverse nature of log data. Large Language Models (LLMs) such as
BERT and GPT3 are trained using self-supervision on a vast amount of unlabeled
data. These models provide generalized representations that can be effectively
used for various downstream tasks with limited labelled data. Motivated by the
success of LLMs in specific domains like science and biology, this paper
introduces a LLM for log data which is trained on public and proprietary log
data. The results of our experiments demonstrate that the proposed LLM
outperforms existing models on multiple downstream tasks. In summary, AIOps
powered by LLMs offers an efficient and effective solution for automating log
analysis tasks and enabling SREs to focus on higher-level tasks. Our proposed
LLM, trained on public and proprietary log data, offers superior performance on
multiple downstream tasks, making it a valuable addition to the AIOps platform.
Related papers
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts.
We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM.
We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
arXiv Detail & Related papers (2024-07-16T04:41:58Z) - LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis [32.46940506638522]
We introduce LogEval, a benchmark suite designed to evaluate the capabilities of Large Language Models in log analysis tasks.
This benchmark covers tasks such as log parsing, log anomaly detection, log fault diagnosis, and log summarization.
LogEval evaluates each task using 4,000 publicly available log data entries and employs 15 different prompts for each task to ensure a thorough and fair assessment.
arXiv Detail & Related papers (2024-07-02T02:39:33Z) - DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph [70.79413606968814]
We introduce Dynamic Evaluation of LLMs via Adaptive Reasoning Graph Evolvement (DARG) to dynamically extend current benchmarks with controlled complexity and diversity.
Specifically, we first extract the reasoning graphs of data points in current benchmarks and then perturb the reasoning graphs to generate novel testing data.
Such newly generated test samples can have different levels of complexity while maintaining linguistic diversity similar to the original benchmarks.
arXiv Detail & Related papers (2024-06-25T04:27:53Z) - ULog: Unsupervised Log Parsing with Large Language Models through Log Contrastive Units [34.344687402936835]
We propose ULog, an unsupervised-based method for efficient and off-the-shelf log parsing.
We refer to such groups of logs as Log Contrastive Units (LCUs)
ULog crafts a novel parsing prompt for LLMs to identify contrastive patterns and extract meaningful log structures from LCUs.
arXiv Detail & Related papers (2024-06-11T11:32:01Z) - UniDM: A Unified Framework for Data Manipulation with Large Language Models [66.61466011795798]
Large Language Models (LLMs) resolve multiple data manipulation tasks.
LLMs exhibit bright benefits in terms of performance but still require customized designs to fit each specific task.
We propose UniDM, a unified framework which establishes a new paradigm to process data manipulation tasks.
arXiv Detail & Related papers (2024-05-10T14:44:04Z) - LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection [73.69399219776315]
We propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains.
Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data.
Then, we transfer such knowledge to the target domain via shared parameters.
arXiv Detail & Related papers (2024-01-09T12:55:21Z) - RAPID: Training-free Retrieval-based Log Anomaly Detection with PLM
considering Token-level information [7.861095039299132]
The need for log anomaly detection is growing, especially in real-world applications.
Traditional deep learning-based anomaly detection models require dataset-specific training, leading to corresponding delays.
We introduce RAPID, a model that capitalizes on the inherent features of log data to enable anomaly detection without training delays.
arXiv Detail & Related papers (2023-11-09T06:11:44Z) - SEED: Domain-Specific Data Curation With Large Language Models [22.54280367957015]
We present SEED, an LLM-as-compiler approach that automatically generates domain-specific data curation solutions via Large Language Models (LLMs)
SEED features an that automatically selects from the four LLM-assisted modules and forms a hybrid execution pipeline that best fits the task at hand.
arXiv Detail & Related papers (2023-10-01T17:59:20Z) - On the Effectiveness of Log Representation for Log-based Anomaly Detection [12.980238412281471]
This work investigates and compares the commonly adopted log representation techniques from previous log analysis research.
We select six log representation techniques and evaluate them with seven ML models and four public log datasets.
We also examine the impacts of the log parsing process and the different feature aggregation approaches when they are employed with log representation techniques.
arXiv Detail & Related papers (2023-08-17T02:18:59Z) - LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak
Supervision [63.08516384181491]
We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts.
Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect.
Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
arXiv Detail & Related papers (2021-11-02T15:16:08Z) - Self-Supervised Log Parsing [59.04636530383049]
Large-scale software systems generate massive volumes of semi-structured log records.
Existing approaches rely on log-specifics or manual rule extraction.
We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
arXiv Detail & Related papers (2020-03-17T19:25:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.