Go Static: Contextualized Logging Statement Generation
- URL: http://arxiv.org/abs/2402.12958v1
- Date: Tue, 20 Feb 2024 12:22:59 GMT
- Title: Go Static: Contextualized Logging Statement Generation
- Authors: Yichen Li, Yintong Huo, Renyi Zhong, Zhihan Jiang, Jinyang Liu, Junjie
Huang, Jiazhen Gu, Pinjia He, Michael R.Lyu
- Abstract summary: SCLogger is a contextualized logging statement generation approach with inter-method static contexts.
SCLogger surpasses the state-of-the-art approach by 8.7% in logging position accuracy, 32.1% in level accuracy, 19.6% in variable precision, and 138.4% in text BLEU-4 score.
- Score: 38.15795803230719
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Logging practices have been extensively investigated to assist developers in
writing appropriate logging statements for documenting software behaviors.
Although numerous automatic logging approaches have been proposed, their
performance remains unsatisfactory due to the constraint of the single-method
input, without informative programming context outside the method.
Specifically, we identify three inherent limitations with single-method
context: limited static scope of logging statements, inconsistent logging
styles, and missing type information of logging variables. To tackle these
limitations, we propose SCLogger, the first contextualized logging statement
generation approach with inter-method static contexts. First, SCLogger extracts
inter-method contexts with static analysis to construct the contextualized
prompt for language models to generate a tentative logging statement. The
contextualized prompt consists of an extended static scope and sampled similar
methods, ordered by the chain-of-thought (COT) strategy. Second, SCLogger
refines the access of logging variables by formulating a new refinement prompt
for language models, which incorporates detailed type information of variables
in the tentative logging statement. The evaluation results show that SCLogger
surpasses the state-of-the-art approach by 8.7% in logging position accuracy,
32.1% in level accuracy, 19.6% in variable precision, and 138.4% in text BLEU-4
score. Furthermore, SCLogger consistently boosts the performance of logging
statement generation across a range of large language models, thereby
showcasing the generalizability of this approach.
Related papers
- LogLLM: Log-based Anomaly Detection Using Large Language Models [8.03646578793411]
We propose LogLLM, a log-based anomaly detection framework that leverages large language models (LLMs)
LogLLM employs BERT for extracting semantic vectors from log messages, while utilizing Llama, a transformer decoder-based model, for classifying log sequences.
Our framework is trained through a novel three-stage procedure designed to enhance performance and adaptability.
arXiv Detail & Related papers (2024-11-13T12:18:00Z) - LibreLog: Accurate and Efficient Unsupervised Log Parsing Using Open-Source Large Language Models [3.7960472831772774]
This paper introduces LibreLog, an unsupervised log parsing approach that enhances privacy and reduces operational costs while achieving state-of-the-art parsing accuracy.
Our evaluation on LogHub-2.0 shows that LibreLog achieves 25% higher parsing accuracy and processes 2.7 times faster compared to state-of-the-art LLMs.
arXiv Detail & Related papers (2024-08-02T21:54:13Z) - Log Probabilities Are a Reliable Estimate of Semantic Plausibility in Base and Instruction-Tuned Language Models [50.15455336684986]
We evaluate the effectiveness of LogProbs and basic prompting to measure semantic plausibility.
We find that LogProbs offers a more reliable measure of semantic plausibility than direct zero-shot prompting.
We conclude that, even in the era of prompt-based evaluations, LogProbs constitute a useful metric of semantic plausibility.
arXiv Detail & Related papers (2024-03-21T22:08:44Z) - LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection [73.69399219776315]
We propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains.
Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data.
Then, we transfer such knowledge to the target domain via shared parameters.
arXiv Detail & Related papers (2024-01-09T12:55:21Z) - Log Statements Generation via Deep Learning: Widening the Support
Provided to Developers [16.079459379684554]
LANCE is an approach rooted in deep learning (DL) that has demonstrated the ability to correctly inject a log statement into Java methods.
We present LEONID, a DL-based technique that can distinguish between methods that do and do not require the inclusion of log statements.
arXiv Detail & Related papers (2023-11-08T10:31:18Z) - Data-Driven Approach for Log Instruction Quality Assessment [59.04636530383049]
There are no widely adopted guidelines on how to write log instructions with good quality properties.
We identify two quality properties: 1) correct log level assignment assessing the correctness of the log level, and 2) sufficient linguistic structure assessing the minimal richness of the static text necessary for verbose event description.
Our approach correctly assesses log level assignments with an accuracy of 0.88, and the sufficient linguistic structure with an F1 score of 0.99, outperforming the baselines.
arXiv Detail & Related papers (2022-04-06T07:02:23Z) - In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST)
A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates.
This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
arXiv Detail & Related papers (2022-03-16T11:58:24Z) - Borrowing from Similar Code: A Deep Learning NLP-Based Approach for Log
Statement Automation [0.0]
We introduce an updated and improved log-aware code-clone detection method to predict the location of logging statements.
We incorporate natural language processing (NLP) and deep learning methods to automate the log statements' description prediction.
Our analysis shows that our hybrid NLP and code-clone detection approach (NLP CC'd) outperforms conventional clone detectors in finding log statement locations.
arXiv Detail & Related papers (2021-12-02T14:03:49Z) - Leveraging Code Clones and Natural Language Processing for Log Statement
Prediction [0.0]
This research aims to predict the log statements by utilizing source code clones and natural language processing (NLP)
Our work demonstrates the effectiveness of log-aware clone detection for automated log location and description prediction.
arXiv Detail & Related papers (2021-09-08T18:17:45Z) - Self-Supervised Log Parsing [59.04636530383049]
Large-scale software systems generate massive volumes of semi-structured log records.
Existing approaches rely on log-specifics or manual rule extraction.
We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
arXiv Detail & Related papers (2020-03-17T19:25:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.