Empirical Characterization of Logging Smells in Machine Learning Code

• URL: http://arxiv.org/abs/2603.23769v1
• Date: Tue, 24 Mar 2026 23:01:40 GMT
• Title: Empirical Characterization of Logging Smells in Machine Learning Code
• Abstract summary: We present an empirical study of logging smells in machine learning (ML) projects.<n>We identify 12 categories of logging smells spanning security, metric management, configuration, verbosity, and context-related issues.<n>Our results show that logging smells are widespread in ML systems and vary in frequency and manifestation across projects.
• Score: 7.520925824033758
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Logging plays a central role in ensuring reproducibility, observability, and reliability in machine learning (ML) systems. While logging is generally considered a good engineering practice, poorly designed logging can negatively affect experiment tracking, security, debugging, and system performance. In this paper, we present an empirical study of logging smells in ML projects and propose a taxonomy of ML-specific logging smell types. We conducted a large-scale analysis of 444 ML repositories and manually labeled 2,448 instances of logging smells. Based on this analysis, we identified 12 categories of logging smells spanning security, metric management, configuration, verbosity, and context-related issues. Our results show that logging smells are widespread in ML systems and vary in frequency and manifestation across projects. To assess practical relevance, we conducted a survey with 27 ML practitioners. Most respondents agreed with the identified smells and reported that several types, including Logging Sensitive Data, Metric Overwrite, Missing Hyperparameter Logging, and Log Without Context, have a strong impact on reproducibility, maintainability, and trustworthiness. Other smells, such as Heavy Data Logging and Print-based Logging, were perceived as more context-dependent. We publicly release our labeled dataset to support future research. Our findings highlight logging quality as a critical and underexplored aspect of ML system engineering and open opportunities for automated detection and repair of logging issues.

Related papers

• Empirical Characterization of Logging Smells in Machine Learning Code [7.520925824033758]
  This study aims to empirically identify and characterize logging smells in machine learning systems.<n>We propose to conduct a large-scale mining of open-source ML repositories hosted on GitHub to catalogue recurring logging smells.
  arXiv  Detail & Related papers  (2026-01-09T05:48:42Z)
• Automated File-Level Logging Generation for Machine Learning Applications using LLMs: A Case Study using GPT-4o Mini [3.076436880934678]
  We evaluate the capacity of GPT-4o mini to generate log statements for machine learning projects at file level.<n>We find that the LLM introduces logs in the same place as humans in 63.91% of cases, but at the cost of a high overlogging rate of 82.66%.
  arXiv  Detail & Related papers  (2025-08-06T18:57:51Z)
• MLScent A tool for Anti-pattern detection in ML projects [5.669063174637433]
  This paper introduces MLScent, a novel static analysis tool for code smell detection.<n>MLScent implements 76 distinct detectors across major machine learning frameworks.<n>Results show high accuracy in framework-specific anti-patterns, data handling issues, and general ML code smells.
  arXiv  Detail & Related papers  (2025-01-30T11:19:16Z)
• Taxonomy of Software Log Smells [5.810137118676013]
  The goal of this study is to create a taxonomy of log smells that can help developers write better logging code.<n>We identified logging issues and tools by conducting a survey of the scientific literature.<n>The paper presents a taxonomy of ten log smells, describing several facets for each of them.
  arXiv  Detail & Related papers  (2024-12-12T13:54:17Z)
• Studying and Benchmarking Large Language Models For Log Level Suggestion [49.176736212364496]
  Large Language Models (LLMs) have become a focal point of research across various domains. This paper investigates the impact of characteristics and learning paradigms on the performance of 12 open-source LLMs in log level suggestion.
  arXiv  Detail & Related papers  (2024-10-11T03:52:17Z)
• LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection [73.69399219776315]
  We propose a unified Transformer-based framework for Log anomaly detection (LogFormer) to improve the generalization ability across different domains. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters.
  arXiv  Detail & Related papers  (2024-01-09T12:55:21Z)
• A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We? [42.56249610409624]
  We provide a new collection of annotated log datasets, denoted Loghub-2.0, which can better reflect the characteristics of log data in real-world software systems. We conduct a thorough re-evaluation of 15 state-of-the-art logs in a more rigorous and practical setting. Particularly, we introduce a new evaluation metric to mitigate the sensitivity of existing metrics to imbalanced data distributions.
  arXiv  Detail & Related papers  (2023-08-21T16:24:15Z)
• Log Parsing Evaluation in the Era of Modern Software Systems [47.370291246632114]
  We focus on one integral part of automated log analysis, log parsing, which is the prerequisite to deriving any insights from logs. Our investigation reveals problematic aspects within the log parsing field, particularly its inefficiency in handling heterogeneous real-world logs. We propose a tool, Logchimera, that enables estimating log parsing performance in industry contexts.
  arXiv  Detail & Related papers  (2023-08-17T14:19:22Z)
• LogLAB: Attention-Based Labeling of Log Data Anomalies via Weak Supervision [63.08516384181491]
  We present LogLAB, a novel modeling approach for automated labeling of log messages without requiring manual work by experts. Our method relies on estimated failure time windows provided by monitoring systems to produce precise labeled datasets in retrospect. Our evaluation shows that LogLAB consistently outperforms nine benchmark approaches across three different datasets and maintains an F1-score of more than 0.98 even at large failure time windows.
  arXiv  Detail & Related papers  (2021-11-02T15:16:08Z)
• Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models [59.04636530383049]
  Anomalies or failures in large computer systems, such as the cloud, have an impact on a large number of users. We propose a framework for anomaly detection in log data, as a major troubleshooting source of system information.
  arXiv  Detail & Related papers  (2021-02-23T09:17:05Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.