SACS: A Code Smell Dataset using Semi-automatic Generation Approach

• URL: http://arxiv.org/abs/2602.15342v1
• Date: Tue, 17 Feb 2026 04:15:22 GMT
• Title: SACS: A Code Smell Dataset using Semi-automatic Generation Approach
• Authors: Hanyu Zhang, Tomoji Kishi,
• Abstract summary: Code smell is a great challenge in software, which indicates latent design or implementation flaws.<n>One of the biggest challenges to apply machine learning-technique is the lack of high-quality code smell datasets.<n>In this study, we explore a semi-automatic approach to generate a code smell dataset with high quality data samples.
• Score: 7.718926822172738
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code smell is a great challenge in software refactoring, which indicates latent design or implementation flaws that may degrade the software maintainability and evolution. Over the past of decades, the research on code smell has received extensive attention. Especially the researches applied machine learning-technique have become a popular topic in recent studies. However, one of the biggest challenges to apply machine learning-technique is the lack of high-quality code smell datasets. Manually constructing such datasets is extremely labor-intensive, as identifying code smells requires substantial development expertise and considerable time investment. In contrast, automatically generated datasets, while scalable, frequently exhibit reduced label reliability and compromised data quality. To overcome this challenge, in this study, we explore a semi-automatic approach to generate a code smell dataset with high quality data samples. Specifically, we first applied a set of automatic generation rules to produce candidate smelly samples. We then employed multiple metrics to group the data samples into an automatically accepted group and a manually reviewed group, enabling reviewers to concentrate their efforts on ambiguous samples. Furthermore, we established structured review guidelines and developed a annotation tool to support the manual validation process. Based on the proposed semi-automatic generation approach, we created an open-source code smell dataset, SACS, covering three widely studied code smells: Long Method, Large Class, and Feature Envy. Each code smell category includes over 10,000 labeled samples. This dataset could provide a large-scale and publicly available benchmark to facilitate future studies on code smell detection and automated refactoring.

Related papers

• From Failure to Mastery: Generating Hard Samples for Tool-use Agents [40.331752086107265]
  HardGen is an automatic agentic pipeline designed to generate hard tool-use training samples with verifiable reasoning.<n>The advanced tools and hard queries enable the generation of verifiable complex Chain-of-Thought (CoT)<n>Our code, models, and dataset will be open-sourced to facilitate future research.
  arXiv  Detail & Related papers  (2026-01-04T11:56:33Z)
• OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs [62.68905180014956]
  We introduce OpenCodeInstruct, the largest open-access instruction tuning dataset, comprising 5 million diverse samples.<n>Each sample includes a programming question, solution, test cases, execution feedback, and LLM-generated quality assessments.<n>We fine-tune various base models, including LLaMA and Qwen, across multiple scales (1B+, 3B+, and 7B+) using our dataset.
  arXiv  Detail & Related papers  (2025-04-05T02:52:16Z)
• UnitCoder: Scalable Iterative Code Synthesis with Unit Test Guidance [65.01483640267885]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in various tasks, yet code generation remains a major challenge.<n>We introduce UnitCoder, a systematic pipeline leveraging model-generated unit tests to guide and validate the code generation process.<n>Our work presents a scalable approach that leverages model-generated unit tests to guide the synthesis of high-quality code data from pre-training corpora.
  arXiv  Detail & Related papers  (2025-02-17T05:37:02Z)
• SnipGen: A Mining Repository Framework for Evaluating LLMs for Code [51.07471575337676]
  Language Models (LLMs) are trained on extensive datasets that include code repositories.<n> evaluating their effectiveness poses significant challenges due to the potential overlap between the datasets used for training and those employed for evaluation.<n>We introduce SnipGen, a comprehensive repository mining framework designed to leverage prompt engineering across various downstream tasks for code generation.
  arXiv  Detail & Related papers  (2025-02-10T21:28:15Z)
• EnseSmells: Deep ensemble and programming language models for automated code smells detection [3.974095344344234]
  A smell in software source code denotes an indication of suboptimal design and implementation decisions.<n>This paper proposes a novel approach to code smell detection, constructing a deep learning architecture that places importance on the fusion of structural features and statistical semantics.
  arXiv  Detail & Related papers  (2025-02-07T15:35:19Z)
• How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study [45.126233498200534]
  We introduce CodeSmellEval, a benchmark designed to evaluate the propensity of Large Language Models for generating code smells.<n>Our benchmark includes a novel metric: Propensity Smelly Score (PSC), and a curated dataset of method-level code smells: CodeSmellData.<n>To demonstrate the use of CodeSmellEval, we conducted a case study with two state-of-the-art LLMs, CodeLlama and Mistral.
  arXiv  Detail & Related papers  (2024-12-25T21:56:35Z)
• Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models [81.27391252152199]
  Large language models (LLMs) have achieved impressive performance across various natural language benchmarks. We propose to automate dataset updating and provide systematic analysis regarding its effectiveness. There are two updating strategies: 1) mimicking strategy to generate similar samples based on original data, and 2) extending strategy that further expands existing samples.
  arXiv  Detail & Related papers  (2024-02-19T07:15:59Z)
• PEOPL: Characterizing Privately Encoded Open Datasets with Public Labels [59.66777287810985]
  We introduce information-theoretic scores for privacy and utility, which quantify the average performance of an unfaithful user. We then theoretically characterize primitives in building families of encoding schemes that motivate the use of random deep neural networks.
  arXiv  Detail & Related papers  (2023-03-31T18:03:53Z)
• DACOS-A Manually Annotated Dataset of Code Smells [4.753388560240438]
  We present DACOS, a manually annotated dataset containing 10,267 annotations for 5,192 code snippets. The dataset targets three kinds of code smells at different granularity: multifaceted abstraction, complex method, and long parameter list. We have developed TagMan, a web application to help annotators view and mark the snippets one-by-one and record the provided annotations.
  arXiv  Detail & Related papers  (2023-03-15T16:13:40Z)
• An Empirical Study on Predictability of Software Code Smell Using Deep Learning Models [3.2973778921083357]
  Code smell is a surface indication of something tainted but in terms of software writing practices. Recent studies have observed that codes having code smells are often prone to a higher probability of change in the software development cycle. We developed code smell prediction models with the help of features extracted from source code to predict eight types of code smell.
  arXiv  Detail & Related papers  (2021-08-08T12:36:23Z)
• Learning to Count in the Crowd from Limited Labeled Data [109.2954525909007]
  We focus on reducing the annotation efforts by learning to count in the crowd from limited number of labeled samples. Specifically, we propose a Gaussian Process-based iterative learning mechanism that involves estimation of pseudo-ground truth for the unlabeled data.
  arXiv  Detail & Related papers  (2020-07-07T04:17:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.