Specification and Detection of LLM Code Smells

• URL: http://arxiv.org/abs/2512.18020v1
• Date: Fri, 19 Dec 2025 19:24:56 GMT
• Title: Specification and Detection of LLM Code Smells
• Authors: Brahim Mahmoudi, Zacharie Chenail-Larcher, Naouel Moha, Quentin Stievenert, Florent Avellaneda,
• Abstract summary: We introduce the concept of LLM code smells and formalize five problematic coding practices related to LLM inference in software systems.<n>We extend the detection tool SpecDetect4AI to cover the newly defined LLM code smells and use it to validate their prevalence in a dataset of 200 open-source LLM systems.
• Score: 3.53563608080816
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) have gained massive popularity in recent years and are increasingly integrated into software systems for diverse purposes. However, poorly integrating them in source code may undermine software system quality. Yet, to our knowledge, there is no formal catalog of code smells specific to coding practices for LLM inference. In this paper, we introduce the concept of LLM code smells and formalize five recurrent problematic coding practices related to LLM inference in software systems, based on relevant literature. We extend the detection tool SpecDetect4AI to cover the newly defined LLM code smells and use it to validate their prevalence in a dataset of 200 open-source LLM systems. Our results show that LLM code smells affect 60.50% of the analyzed systems, with a detection precision of 86.06%.

Related papers

• Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection [0.5249836059995157]
  Code smells are symptoms of potential code quality problems that may affect software maintainability.<n>This paper evaluates the effectiveness of four large language models (LLMs) for detecting nine code smells across 30 Java projects.
  arXiv  Detail & Related papers  (2026-01-14T21:08:35Z)
• Investigating The Smells of LLM Generated Code [2.9232837969697965]
  Large Language Models (LLMs) are increasingly being used to generate program code.<n>This study proposes a scenario-based method of evaluating the quality of LLM-generated code.
  arXiv  Detail & Related papers  (2025-10-03T14:09:55Z)
• How Accurately Do Large Language Models Understand Code? [4.817546726074033]
  Large Language Models (LLMs) are increasingly used in post-development tasks such as code repair and testing.<n> Quantifying code comprehension is challenging due to its abstract nature and the lack of a standardized metric.<n>This paper presents the first large-scale empirical investigation into LLMs' ability to understand code.
  arXiv  Detail & Related papers  (2025-04-06T05:59:29Z)
• ExeCoder: Empowering Large Language Models with Executability Representation for Code Translation [57.604506522287814]
  Existing large language models (LLMs) only learn the contextual semantics of code during pre-training.<n>We propose ExeCoder to utilize executability representations such as functional semantics, syntax structures, and variable dependencies.<n>We show that ExeCoder achieves state-of-the-art performance in code translation, surpassing existing open-source code LLMs by over 10.88% to 38.78% and over 27.44% to 42.97% on two metrics.
  arXiv  Detail & Related papers  (2025-01-30T16:18:52Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [76.59316249991657]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems.<n>While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited.<n>We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• Combining LLM Code Generation with Formal Specifications and Reactive Program Synthesis [0.7580487359358722]
  Large Language Models (LLMs) struggle with accuracy and are unsuitable for high-risk applications. We introduce a solution that divides the code generation into two parts; one to be handled by an LLM and one to be handled by formal methods-based program synthesis.
  arXiv  Detail & Related papers  (2024-09-18T15:59:06Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models [10.519984835232359]
  Large Language Models (LLMs) have demonstrated unprecedented capabilities in code generation.<n>We conducted an in-depth analysis of code generation errors across six representative LLMs on the HumanEval dataset.<n>Our findings highlighted several challenges in locating and fixing code generation errors made by LLMs.
  arXiv  Detail & Related papers  (2024-06-13T01:29:52Z)
• InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models [56.723509505549536]
  InfiBench is the first large-scale freeform question-answering (QA) benchmark for code to our knowledge. It comprises 234 carefully selected high-quality Stack Overflow questions that span across 15 programming languages. We conduct a systematic evaluation for over 100 latest code LLMs on InfiBench, leading to a series of novel and insightful findings.
  arXiv  Detail & Related papers  (2024-03-11T02:06:30Z)
• Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs [60.40396361115776]
  This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in large language models (LLMs) with a slim proxy model. We employ a proxy model which has far fewer parameters, and take its answers as answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM.
  arXiv  Detail & Related papers  (2024-02-19T11:11:08Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.