A Defect Taxonomy for Infrastructure as Code: A Replication Study

• URL: http://arxiv.org/abs/2505.01568v3
• Date: Wed, 18 Jun 2025 13:00:14 GMT
• Title: A Defect Taxonomy for Infrastructure as Code: A Replication Study
• Abstract summary: Prior study developed such a taxonomy, but based it exclusively on the declarative language Puppet.<n>It remained unknown whether this taxonomy applies to programming language-based IaC tools like Pulumi, Terraform CDK, and AWS CDK.<n>We performed qualitative analysis on 3,364 defect-related commits from 285 open-source PL-IaC repositories.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: As Infrastructure as Code (IaC) becomes standard practice, ensuring the reliability of IaC scripts is essential. Defect taxonomies are valuable tools for this, offering a common language for issues and enabling systematic tracking. A significant prior study developed such a taxonomy, but based it exclusively on the declarative language Puppet. It remained unknown whether this taxonomy applies to programming language-based IaC (PL-IaC) tools like Pulumi, Terraform CDK, and AWS CDK. Aim: We replicated this foundational work to assess the generalizability of the taxonomy across a broader and more diverse landscape. Method: We performed qualitative analysis on 3,364 defect-related commits from 285 open-source PL-IaC repositories (PIPr dataset) to derive a PL-IaC-specific defect taxonomy. We then enhanced the ACID tool, originally developed for the prior study, to automatically classify and analyze defect distributions across an expanded dataset-447 open-source repositories and 94 proprietary projects from VTEX (e-commerce) and Nubank (financial). Results: Our research confirmed the same eight defect categories identified in the original study, with idempotency and security defects appearing infrequently but persistently across projects. Configuration Data defects maintain high frequency in both open-source and proprietary codebases. Conclusions: Our replication supports the generalizability of the original taxonomy, suggesting IaC development challenges surpass organizational boundaries. Configuration Data defects emerge as a persistent high-frequency problem, while idempotency and security defects remain important concerns despite lower frequency. These patterns appear consistent across open-source and proprietary projects, indicating they are fundamental to the IaC paradigm itself, transcending specific tools or project types.

Related papers

• Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection [1.9637266103693347]
  We propose a nine-dimensional quality evaluation metric consisting of three categories: structural, behavioral, and semantic similarity.<n>Vulnerabilities often arise in details that are easily lost in this process, such as error-handling flows and boundary checks.<n>This study provides a statistical evaluation foundation for quantifying implementation defects in IoT devices.
  arXiv  Detail & Related papers  (2026-08-07T08:36:35Z)
• SWE-InfraBench: Evaluating Language Models on Cloud Infrastructure Code [0.6695324715259999]
  Building infrastructure-as-code (IaC) in cloud computing is a critical task.<n>Our benchmark reflects the incremental code edits common in enterprise development with imperative tools like the AWS CDK.<n>We present SWE-InfraBench, a diverse evaluation dataset sourced from dozens of real-world IaCs.
  arXiv  Detail & Related papers  (2026-06-03T12:48:53Z)
• GCVE: A Decentralized Model for Vulnerability Identification, Publication, and Operational Enrichment [51.77969450792284]
  The Global CVE initiative (G) proposes a decentralized, open, and identifier model for vulnerability identification, publication, and enrichment.<n>This paper presents G as socio-technical effort combining autonomous G Numbering Authorities, lightweight allocation rules, distributed publication, open Best Current Practices, and practical reference implementations.
  arXiv  Detail & Related papers  (2026-05-30T19:03:39Z)
• What Makes Software Bugs Escape Testing? Evidence from a Large-Scale Empirical Study [2.8358949816586025]
  This paper presents a large-scale characterization of pre- and post-release defects across C/C++ and Java systems.<n>Results show that post-release defects are concentrated in older, frequently modified, and high-churn components.
  arXiv  Detail & Related papers  (2026-04-29T13:42:25Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Architecture-Aware Multi-Design Generation for Repository-Level Feature Addition [53.50448142467294]
  RAIM is a multi-design and architecture-aware framework for repository-level feature addition.<n>It shifts away from linear patching by generating multiple diverse implementation designs.<n>Experiments on the NoCode-bench Verified dataset demonstrate that RAIM establishes a new state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-03-02T12:50:40Z)
• Bridging the Arithmetic Gap: The Cognitive Complexity Benchmark and Financial-PoT for Robust Financial Reasoning [11.522192050185568]
  Large Language Models suffer from "Arithmetic Hallucinations" and a systemic failure mode we term "Cognitive Collapse"<n>We introduce the Cognitive Complexity Benchmark (CCB), a robust evaluation framework grounded in a dataset constructed from 95 real-world Chinese A-share annual reports.<n>We propose the Iterative Dual-Phase Financial-PoT framework to address these failures.
  arXiv  Detail & Related papers  (2026-01-29T01:33:33Z)
• Characterizing Bugs and Quality Attributes in Quantum Software: A Large-Scale Empirical Study [0.6445605125467574]
  This study presents the first ecosystem-scale longitudinal analysis of software bugs across 123 open source quantum repositories from 2012 to 2024.<n>Full-stack libraries and compilers are the most bug-prone categories due to circuit, gate, and transpilation-related issues.<n>High-severity bugs cluster in cryptography, experimental computing, and compiler toolchains.
  arXiv  Detail & Related papers  (2025-12-31T06:05:49Z)
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning [65.20602712957725]
  Caco is a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-CoT reasoning data.<n>Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.
  arXiv  Detail & Related papers  (2025-10-05T07:59:24Z)
• Adapting Vision-Language Models Without Labels: A Comprehensive Survey [74.17944178027015]
  Vision-Language Models (VLMs) have demonstrated remarkable generalization capabilities across a wide range of tasks.<n>Recent research has increasingly focused on unsupervised adaptation methods that do not rely on labeled data.<n>We propose a taxonomy based on the availability and nature of unlabeled visual data, categorizing existing approaches into four key paradigms.
  arXiv  Detail & Related papers  (2025-08-07T16:27:37Z)
• Refactoring $\ eq$ Bug-Inducing: Improving Defect Prediction with Code Change Tactics Analysis [54.361900378970134]
  Just-in-time defect prediction (JIT-DP) aims to predict the likelihood of code changes resulting in software defects at an early stage.<n>Prior research has largely ignored code during both the evaluation and methodology phases, despite its prevalence.<n>We propose Code chAnge Tactics (CAT) analysis to categorize code and its propagation, which improves labeling accuracy in the JIT-Defects4J dataset by 13.7%.
  arXiv  Detail & Related papers  (2025-07-25T23:29:25Z)
• FORGE: An LLM-driven Framework for Large-Scale Smart Contract Vulnerability Dataset Construction [34.20628333535654]
  FORGE is the first automated approach for constructing smart contract vulnerability datasets.<n>We generate a dataset comprising 81,390 solidity files and 27,497 vulnerability findings across 296 CWE categories.<n>Results reveal the significant limitations in current detection capabilities.
  arXiv  Detail & Related papers  (2025-06-23T16:03:16Z)
• Boosting Vulnerability Detection of LLMs via Curriculum Preference Optimization with Synthetic Reasoning Data [22.557961978833386]
  We propose a novel framework for large language models (LLMs) that excels at mining vulnerability patterns.<n>Specifically, we construct forward and backward reasoning processes for vulnerability and corresponding fixed code, ensuring the synthesis of high-quality reasoning data.<n>We show that ReVD sets new state-of-the-art for LLM-based software vulnerability detection, e.g., 12.24%-22.77% improvement in the accuracy.
  arXiv  Detail & Related papers  (2025-06-09T03:25:23Z)
• Smells-sus: Sustainability Smells in IaC [0.0]
  This research focuses on Terraform, a widely adopted Infrastructure as Code (IaC) tool.<n>We define seven sustainability smells and validate them through a survey with 19 IaC practitioners.<n>The most prevalent smell is Monolithic Infrastructure, which appears in 9.67% of the scripts.
  arXiv  Detail & Related papers  (2025-01-13T20:24:10Z)
• KBAlign: Efficient Self Adaptation on Specific Knowledge Bases [73.34893326181046]
  We present KBAlign, a self-supervised framework that enhances RAG systems through efficient model adaptation.<n>Our key insight is to leverage the model's intrinsic capabilities for knowledge alignment through two innovative mechanisms.<n> Experiments demonstrate that KBAlign can achieve 90% of the performance gain obtained through GPT-4-supervised adaptation.
  arXiv  Detail & Related papers  (2024-11-22T08:21:03Z)
• Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs [49.57641083688934]
  We introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines.
  arXiv  Detail & Related papers  (2024-06-05T20:19:09Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Unveiling Hidden Links Between Unseen Security Entities [3.7138962865789353]
  VulnScopper is an innovative approach that utilizes multi-modal representation learning, combining Knowledge Graphs (KG) and Natural Processing (NLP) We evaluate VulnScopper on two major security datasets, the National Vulnerability Database (NVD) and the Red Hat CVE database. Our results show that VulnScopper outperforms existing methods, achieving up to 78% Hits@10 accuracy in linking CVEs to Common Vulnerabilities and Exposures (CWEs), and Common Platform Languageions (CPEs)
  arXiv  Detail & Related papers  (2024-03-04T13:14:39Z)
• Patch2QL: Discover Cognate Defects in Open Source Software Supply Chain With Auto-generated Static Analysis Rules [1.9591497166224197]
  We propose a novel technique for detecting cognate defects in OSS through the automatic generation of SAST rules. Specifically, it extracts key syntax and semantic information from pre- and post-patch versions of code. We have implemented a prototype tool called Patch2QL and applied it to fundamental OSS in C/C++.
  arXiv  Detail & Related papers  (2024-01-23T02:23:11Z)
• Geometric Deep Learning for Structure-Based Drug Design: A Survey [83.87489798671155]
  Structure-based drug design (SBDD) leverages the three-dimensional geometry of proteins to identify potential drug candidates. Recent advancements in geometric deep learning, which effectively integrate and process 3D geometric data, have significantly propelled the field forward.
  arXiv  Detail & Related papers  (2023-06-20T14:21:58Z)
• On the Security Blind Spots of Software Composition Analysis [46.1389163921338]
  We present a novel approach to detect vulnerable clones in the Maven repository. We retrieve over 53k potential vulnerable clones from Maven Central. We detect 727 confirmed vulnerable clones and synthesize a testable proof-of-vulnerability project for each of those.
  arXiv  Detail & Related papers  (2023-06-08T20:14:46Z)
• Upcycling Models under Domain and Category Shift [95.22147885947732]
  We introduce an innovative global and local clustering learning technique (GLC) We design a novel, adaptive one-vs-all global clustering algorithm to achieve the distinction across different target classes. Remarkably, in the most challenging open-partial-set DA scenario, GLC outperforms UMAD by 14.8% on the VisDA benchmark.
  arXiv  Detail & Related papers  (2023-03-13T13:44:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.