On the Relation between Code Quality and Machine Learning Performance: A Large-scale Empirical Study

• URL: http://arxiv.org/abs/2609.10610v1
• Date: Tue, 08 Sep 2026 12:52:54 GMT
• Title: On the Relation between Code Quality and Machine Learning Performance: A Large-scale Empirical Study
• Abstract summary: We investigated the relationship between code quality and machine learning (ML) performance in notebooks.<n>General Python code quality is decoupled from ML performance, showing negligible or non-significant correlations across all observations.<n>Code expertise provides no information on quality or performance, but competition expertise correlates with better performance.
• Score: 0.13381749415517016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Context: Computational notebooks are the standard environment for machine learning (ML) development. Within the ML community, model performance is often the primary considered metric, and code quality is treated as a secondary concern. This prioritization relies on a largely untested assumption that code quality and ML performance are unrelated. Practitioners also reuse existing code that may come from notebooks selected through social signals (popularity, author expertise) whose reliability as quality proxies has never been assessed. Objective: We empirically investigated the relationship between code quality and ML performance in notebooks, and evaluated whether popularity and author expertise give indication on code quality or performance. Method: We conducted a large-scale empirical study of 265,363 Python notebooks submitted to Kaggle competitions. We assessed code quality with two static analysis tools: Pylint, capturing general Python code quality, and SonarQube, configured with a profile of 34 rules targeting data-science and ML-specific practices. Results: The relationship between code quality and performance depends on the notion of quality considered. General Python code quality is decoupled from ML performance, showing negligible or non-significant correlations across all observations. In contrast, ML-specific violations exhibit a consistent, small negative association with performance that persists across all observations. The popularity of a notebook does not give information on the code quality or performance. Code expertise provides no information on quality or performance, but competition expertise correlates with better performance, fewer ML-specific violations, and slightly more Python errors and refactoring violations.

Related papers

• From Code Changes to Quality Gains: An Empirical Study in Python ML Systems with PyQu [1.5259434915412646]
  We introduce PyQu, a novel tool to identify quality-enhancing commits with an average accuracy, precision, and recall of 0.84 and 0.85 of average F1 score.<n> 41% of the changes are newly discovered by our study and have not been identified by state-of-the-art Python changes detection tools.
  arXiv  Detail & Related papers  (2025-11-04T18:55:19Z)
• Analysing Python Machine Learning Notebooks with Moose [0.12249546377051435]
  Machine Learning (ML) code, particularly within notebooks, often exhibits lower quality compared to traditional software.<n>Bad practices arise at three distinct levels: general Python coding conventions, the organizational structure of the notebook itself, and ML-specific aspects such as API usage.<n>This paper introduces Vespucci Linter, a static analysis tool with multi-level capabilities, built on Moose.
  arXiv  Detail & Related papers  (2025-09-15T09:59:49Z)
• CoQuIR: A Comprehensive Benchmark for Code Quality-Aware Information Retrieval [44.39462689088926]
  CoQuIR is the first large-scale, multilingual benchmark designed to evaluate quality-aware code retrieval.<n>CoQuIR provides fine-grained quality annotations for 42,725 queries and 134,907 code snippets in 11 programming languages.
  arXiv  Detail & Related papers  (2025-05-31T13:00:17Z)
• Is Compression Really Linear with Code Intelligence? [60.123628177110206]
  textitFormat Annealing is a lightweight, transparent training methodology designed to assess the intrinsic capabilities of pre-trained models equitably.<n>Our empirical results reveal a fundamental logarithmic relationship between measured code intelligence and bits-per-character (BPC)<n>Our work provides a more nuanced understanding of compression's role in developing code intelligence and contributes a robust evaluation framework in the code domain.
  arXiv  Detail & Related papers  (2025-05-16T16:59:14Z)
• Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation [69.62857948698436]
  Recent advances in large language models (LLMs) have improved their performance on coding benchmarks.<n>However, improvement is plateauing due to the exhaustion of readily available high-quality data.<n>We propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity.
  arXiv  Detail & Related papers  (2025-02-20T18:32:19Z)
• Evaluating Source Code Quality with Large Languagem Models: a comparative study [2.3204178451683264]
  This paper describes and analyzes the results obtained using Large Language Model (LLM) as a static analysis tool. A total of 1,641 classes were analyzed, comparing the results in two versions of models: GPT 3.5 Turbo and GPT 4o. The GPT 4o version did not present the same results, diverging from the previous model and Sonar by assigning a high classification to codes that were assessed as lower quality.
  arXiv  Detail & Related papers  (2024-08-07T18:44:46Z)
• Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph [83.90988015005934]
  Uncertainty quantification is a key element of machine learning applications.<n>We introduce a novel benchmark that implements a collection of state-of-the-art UQ baselines.<n>We conduct a large-scale empirical investigation of UQ and normalization techniques across eleven tasks, identifying the most effective approaches.
  arXiv  Detail & Related papers  (2024-06-21T20:06:31Z)
• The Fault in our Stars: Quality Assessment of Code Generation Benchmarks [0.5137309756089941]
  We conduct the first-of-its-kind study of the quality of prompts within benchmarks used to compare the performance of different code generation models. We analyzed 3,566 prompts from 9 code generation benchmarks to identify quality issues in them.
  arXiv  Detail & Related papers  (2024-04-15T22:02:58Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)
• Measuring Coding Challenge Competence With APPS [54.22600767666257]
  We introduce APPS, a benchmark for code generation. Our benchmark includes 10,000 problems, which range from having simple one-line solutions to being substantial algorithmic challenges. Recent models such as GPT-Neo can pass approximately 15% of the test cases of introductory problems.
  arXiv  Detail & Related papers  (2021-05-20T17:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.