Reshaping the SDLC for Data- and AI-Centric Systems

• URL: http://arxiv.org/abs/2608.17824v1
• Date: Tue, 18 Aug 2026 14:23:09 GMT
• Title: Reshaping the SDLC for Data- and AI-Centric Systems
• Abstract summary: This paper examines how integrating data engineering and software engineering practices, operationalized through DataOps, MLOps, and LLMOps, reshapes the SDLC for data- and AI-centric systems.<n>First, we synthesize literature across software engineering, data management, machine learning systems, and human-centered computing into a phase-structured account of lifecycle transformation.<n>Second, we provide a lightweight formalization in which system behavior is defined over code, data, and model configurations.<n>Third, we develop an adaptive five-layer lifecycle framework comprising artifact, contract, gate, control, and governance layers
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The traditional Software Development Lifecycle (SDLC) assumes that system behavior is determined primarily by source code, allowing correctness to be specified, implemented, and verified through code-centric practices. Data-intensive and AI-enabled systems challenge this assumption because their behavior emerges from the interaction of code, data, and learned models, while performance may degrade as real-world conditions drift from training data. This paper examines how integrating data engineering and software engineering practices, operationalized through DataOps, MLOps, and LLMOps, reshapes the SDLC for these systems. We make four contributions. First, we synthesize literature across software engineering, data management, machine learning systems, and human-centered computing into a phase-structured account of lifecycle transformation spanning requirements, architecture, development, testing, deployment, monitoring, governance, and organization. Second, we provide a lightweight formalization in which system behavior is defined over code, data, and model configurations; requirements become evaluation-led specifications with probabilistic acceptance regions; and promotion is controlled through statistically grounded validation gates. Third, we develop an adaptive five-layer lifecycle framework comprising artifact, contract, gate, control, and governance layers, positioning maintenance as a closed-loop control problem under configuration drift. Fourth, we propose a conceptual research model linking data engineering integration to measurable lifecycle outcomes and critically assess the evidence base. While the direction of transformation is increasingly established, its magnitude remains insufficiently quantified. We conclude with a research agenda for an empirically grounded, adaptive SDLC for data- and AI-centric systems.

Related papers

• KAPPS: A knowledge-based CPPS Architecture for the Circular Factory [10.732247472523357]
  Circular manufacturing reintroduces used products with heterogeneous and uncertain conditions.<n>This shift demands manufacturing systems capable of handling variable product states, dynamically reconfigurable processes, and the integration of human and machine knowledge.<n>Conventional manufacturing IT architectures, designed for stable structures and deterministic execution, are unable to meet these requirements.
  arXiv  Detail & Related papers  (2026-05-21T13:20:53Z)
• daVinci-LLM:Towards the Science of Pretraining [58.6458452580545]
  daVinci-LLM combines industrial-scale resources with full research freedom to advance the science of pretraining.<n>We release complete data processing pipelines, full training processes, and systematic exploration results.
  arXiv  Detail & Related papers  (2026-03-28T07:00:49Z)
• Data Science and Technology Towards AGI Part I: Tiered Data Management [53.64581824953229]
  We argue that the development of artificial intelligence is entering a new phase of data-model co-evolution.<n>We introduce an L0-L4 tiered data management framework, ranging from raw uncurated resources to organized and verifiable knowledge.<n>We validate the effectiveness of the proposed framework through empirical studies.
  arXiv  Detail & Related papers  (2026-02-09T18:47:51Z)
• Data-Driven Methods and AI in Engineering Design: A Systematic Literature Review Focusing on Challenges and Opportunities [0.2545763876632975]
  Machine learning and statistical methods dominate current practice, whereas deep learning exhibits a clear upward trend in adoption.<n>Key challenges in existing applications include limited model interpretability, poor cross-stage traceability, and insufficient validation under real-world conditions.<n>This review is a first step toward design-stage guidelines; a follow-up synthesis should map computer science algorithms to engineering design problems and activities.
  arXiv  Detail & Related papers  (2025-11-25T11:16:38Z)
• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• VDSAgents: A PCS-Guided Multi-Agent System for Veridical Data Science Automation [11.521235834823301]
  Large language models (LLMs) become increasingly integrated into data science for automated system design.<n>This paper provides VDSAgents, a multi-agent system grounded in the Predictability-Computability-Stability (PCS) principles.<n>We evaluate VDSAgents on nine datasets with diverse characteristics, comparing it with state-of-the-art end-to-end data science systems.
  arXiv  Detail & Related papers  (2025-10-28T12:07:50Z)
• From Physics to Machine Learning and Back: Part II - Learning and Observational Bias in PHM [52.64097278841485]
  Review examines how incorporating learning and observational biases through physics-informed modeling and data strategies can guide models toward physically consistent and reliable predictions.<n>Fast adaptation methods including meta-learning and few-shot learning are reviewed alongside domain generalization techniques.
  arXiv  Detail & Related papers  (2025-09-25T14:15:43Z)
• A Theoretical Framework for AI-driven data quality monitoring in high-volume data environments [1.2753215270475886]
  This paper presents a theoretical framework for an AI-driven data quality monitoring system designed to address the challenges of maintaining data quality in high-volume environments. We examine the limitations of traditional methods in managing the scale, velocity, and variety of big data and propose a conceptual approach leveraging advanced machine learning techniques. Key components include an intelligent data ingestion layer, adaptive preprocessing mechanisms, context-aware feature extraction, and AI-based quality assessment modules.
  arXiv  Detail & Related papers  (2024-10-11T07:06:36Z)
• Integration of Domain Expert-Centric Ontology Design into the CRISP-DM for Cyber-Physical Production Systems [45.05372822216111]
  Methods from Machine Learning (ML) and Data Mining (DM) have proven to be promising in extracting complex and hidden patterns from the data collected. However, such data-driven projects, usually performed with the Cross-Industry Standard Process for Data Mining (CRISPDM), often fail due to the disproportionate amount of time needed for understanding and preparing the data. This contribution intends present an integrated approach so that data scientists are able to more quickly and reliably gain insights into the CPPS challenges.
  arXiv  Detail & Related papers  (2023-07-21T15:04:00Z)
• SOLIS -- The MLOps journey from data acquisition to actionable insights [62.997667081978825]
  In this paper we present a unified deployment pipeline and freedom-to-operate approach that supports all requirements while using basic cross-platform tensor framework and script language engines. This approach however does not supply the needed procedures and pipelines for the actual deployment of machine learning capabilities in real production grade systems.
  arXiv  Detail & Related papers  (2021-12-22T14:45:37Z)
• An Extensible Benchmark Suite for Learning to Simulate Physical Systems [60.249111272844374]
  We introduce a set of benchmark problems to take a step towards unified benchmarks and evaluation protocols. We propose four representative physical systems, as well as a collection of both widely used classical time-based and representative data-driven methods.
  arXiv  Detail & Related papers  (2021-08-09T17:39:09Z)
• From Data to Actions in Intelligent Transportation Systems: a Prescription of Functional Requirements for Model Actionability [10.27718355111707]
  This work aims to describe how data, coming from diverse ITS sources, can be used to learn and adapt data-driven models for efficiently operating ITS assets, systems and processes. Grounded in this described data modeling pipeline for ITS, wedefine the characteristics, engineering requisites and intrinsic challenges to its three compounding stages, namely, data fusion, adaptive learning and model evaluation.
  arXiv  Detail & Related papers  (2020-02-06T12:02:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.