Automatic Pipeline Provisioning

• URL: http://arxiv.org/abs/2511.14825v1
• Date: Tue, 18 Nov 2025 16:59:28 GMT
• Title: Automatic Pipeline Provisioning
• Abstract summary: The goal of this paper is to explore the benefits of automatic pipeline provisioning and identify how it can be applied.<n>This research will focus on CI pipelines, although the outcomes of this approach on CD pipelines will likely be similar.
• Score: 45.88028371034407
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The goal of this paper is to explore the benefits of automatic pipeline provisioning and identify how it can be applied. Automatic pipeline provisioning can be defined as a process of quickly deploying a pipeline for a software engineering project. This research will focus on CI pipelines, although the outcomes of this approach on CD pipelines will likely be similar.

Related papers

• A Unified Experimental Architecture for Informative Path Planning: from Simulation to Deployment with GuadalPlanner [69.43049144653882]
  This paper introduces a unified architecture that decouples high-level decision-making from vehicle-specific control.<n>The proposed architecture is realized through GuadalPlanner, which defines standardized interfaces between planning, sensing, and vehicle execution.
  arXiv  Detail & Related papers  (2026-02-11T10:02:31Z)
• AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models [59.7059443712562]
  AdaPtis is a training system for large language models (LLMs) that supports adaptive pipeline parallelism.<n>Extensive experiments show that AdaPtis achieves an average speedup of 1.42x (up to 2.14x) over Megatron-LM I-1F1B.
  arXiv  Detail & Related papers  (2025-09-28T08:05:13Z)
• Instrumentation and Analysis of Native ML Pipelines via Logical Query Plans [3.2362171533623054]
  We envision highly-automated software platforms to assist data scientists with developing, validating, monitoring, and analysing their Machine Learning pipelines. We extract "logical query plans" from ML pipeline code relying on popular libraries. Based on these plans, we automatically infer pipeline semantics and instrument and rewrite the ML pipelines to enable diverse use cases without requiring data scientists to manually annotate or rewrite their code.
  arXiv  Detail & Related papers  (2024-07-10T11:35:02Z)
• Trusted Provenance of Automated, Collaborative and Adaptive Data Processing Pipelines [2.186901738997927]
  We provide a solution architecture and a proof of concept implementation of a service, called Provenance Holder. Provenance Holder enables provenance of collaborative, adaptive data processing pipelines in a trusted manner.
  arXiv  Detail & Related papers  (2023-10-17T17:52:27Z)
• Deep Pipeline Embeddings for AutoML [11.168121941015015]
  AutoML is a promising direction for democratizing AI by automatically deploying Machine Learning systems with minimal human expertise. Existing Pipeline Optimization techniques fail to explore deep interactions between pipeline stages/components. This paper proposes a novel neural architecture that captures the deep interaction between the components of a Machine Learning pipeline.
  arXiv  Detail & Related papers  (2023-05-23T12:40:38Z)
• Automatic Design Method of Building Pipeline Layout Based on Deep Reinforcement Learning [2.2284196209013345]
  Currently, pipeline layout is designed manually by engineers. We propose a method for generating three-dimensional layout of pipelines based on deep reinforcement learning (DRL) Our results demonstrate that DRL models can complete the pipeline layout task in space in a much shorter time than traditional algorithms.
  arXiv  Detail & Related papers  (2023-05-18T07:01:58Z)
• Towards Personalized Preprocessing Pipeline Search [52.59156206880384]
  ClusterP3S is a novel framework for Personalized Preprocessing Pipeline Search via Clustering. We propose a hierarchical search strategy to jointly learn the clusters and search for the optimal pipelines. Experiments on benchmark classification datasets demonstrate the effectiveness of enabling feature-wise preprocessing pipeline search.
  arXiv  Detail & Related papers  (2023-02-28T05:45:05Z)
• Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines [77.45213180689952]
  Preprocessing pipelines in deep learning aim to provide sufficient data throughput to keep the training processes busy. We introduce a new perspective on efficiently preparing datasets for end-to-end deep learning pipelines. We obtain an increased throughput of 3x to 13x compared to an untuned system.
  arXiv  Detail & Related papers  (2022-02-17T14:31:58Z)
• Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines [48.7576911714538]
  The proposed approach is aimed to automate the design of composite machine learning pipelines. It designs the pipelines with a customizable graph-based structure, analyzes the obtained results, and reproduces them. The software implementation on this approach is presented as an open-source framework.
  arXiv  Detail & Related papers  (2021-06-26T23:19:06Z)
• TODS: An Automated Time Series Outlier Detection System [70.88663649631857]
  TODS is a highly modular system that supports easy pipeline construction.<n>Tods supports 70 primitives, including data processing, time series processing, feature analysis, detection algorithms, and a reinforcement module.
  arXiv  Detail & Related papers  (2020-09-18T15:36:43Z)
• AVATAR -- Machine Learning Pipeline Evaluation Using Surrogate Model [10.83607599315401]
  We propose a novel method to evaluate the validity of ML pipelines using a surrogate model (AVATAR) Our experiments show that the AVATAR is more efficient in evaluating complex pipelines in comparison with the traditional evaluation approaches requiring their execution.
  arXiv  Detail & Related papers  (2020-01-30T02:53:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.