An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows

• URL: http://arxiv.org/abs/2507.18062v1
• Date: Thu, 24 Jul 2025 03:26:38 GMT
• Title: An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows
• Authors: Edward Abrokwah, Taher A. Ghaleb,
• Abstract summary: GitHub Actions (GHA) has emerged as a dominant service due to its deep integration with GitHub.<n>GHA provides official documentation and community-supported best practices.<n>This study will investigate the structure, complexity, and compliance of GHA in open-source software repositories.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continuous Integration (CI) has evolved from a tooling strategy to a fundamental mindset in modern CI engineering. It enables teams to develop, test, and deliver software rapidly and collaboratively. Among CI services, GitHub Actions (GHA) has emerged as a dominant service due to its deep integration with GitHub and a vast ecosystem of reusable workflow actions. Although GHA provides official documentation and community-supported best practices, there appears to be limited empirical understanding of how open-source real-world CI workflows align with such practices. Many workflows might be unnecessarily complex and not aligned with the simplicity goals of CI practices. This study will investigate the structure, complexity, heterogeneity, and compliance of GHA workflows in open-source software repositories. Using a large dataset of GHA workflows from Java, Python, and C++ repositories, our goal is to (a) identify workflow complexities, (b) analyze recurring and heterogeneous structuring patterns, (c) assess compliance with GHA best practices, and (d) uncover differences in CI pipeline design across programming languages. Our findings are expected to reveal both areas of strong adherence to best practices and areas for improvement where needed. These insights will also have implications for CI services, as they will highlight the need for clearer guidelines and comprehensive examples in CI documentation.

Related papers

• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• Benchmarking Agentic Workflow Generation [80.74757493266057]
  We introduce WorfBench, a unified workflow generation benchmark with multi-faceted scenarios and intricate graph workflow structures.<n>We also present WorfEval, a systemic evaluation protocol utilizing subsequence and subgraph matching algorithms.<n>We observe that the generated can enhance downstream tasks, enabling them to achieve superior performance with less time during inference.
  arXiv  Detail & Related papers  (2024-10-10T12:41:19Z)
• Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? [60.84912551069379]
  We present the Code-Development Benchmark (Codev-Bench), a fine-grained, real-world, repository-level, and developer-centric evaluation framework. Codev-Agent is an agent-based system that automates repository crawling, constructs execution environments, extracts dynamic calling chains from existing unit tests, and generates new test samples to avoid data leakage.
  arXiv  Detail & Related papers  (2024-10-02T09:11:10Z)
• The Hidden Costs of Automation: An Empirical Study on GitHub Actions Workflow Maintenance [45.53834452021771]
  GitHub Actions (GA) is an orchestration platform that streamlines the automatic execution of engineering tasks. Human intervention is necessary to correct defects, update dependencies, or existing workflow files.
  arXiv  Detail & Related papers  (2024-09-04T01:33:16Z)
• Automatic Categorization of GitHub Actions with Transformers and Few-shot Learning [12.254055731378045]
  GitHub Actions (GHA) have been conceived to provide developers with a practical tool to create and maintain a pipeline. To expose actions to search engines, GitHub allows developers to assign them to one or more categories manually. We propose Gavel, a practical solution to increasing the visibility of actions in GitHub.
  arXiv  Detail & Related papers  (2024-07-24T02:27:36Z)
• Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration [64.19431011897515]
  This paper presents Alibaba LingmaAgent, a novel Automated Software Engineering method designed to comprehensively understand and utilize whole software repositories for issue resolution.<n>Our approach introduces a top-down method to condense critical repository information into a knowledge graph, reducing complexity, and employs a Monte Carlo tree search based strategy.<n>In production deployment and evaluation at Alibaba Cloud, LingmaAgent automatically resolved 16.9% of in-house issues faced by development engineers, and solved 43.3% of problems after manual intervention.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)
• Detecting Continuous Integration Skip : A Reinforcement Learning-based Approach [0.4297070083645049]
  Continuous Integration (CI) practices facilitate the seamless integration of code changes by employing automated building and testing processes. Some frameworks, such as Travis CI and GitHub Actions have significantly contributed to simplifying and enhancing the CI process. Developers continue to encounter difficulties in accurately flagging commits as either suitable for CI execution or as candidates for skipping.
  arXiv  Detail & Related papers  (2024-05-15T18:48:57Z)
• Leveraging Large Language Models to Build and Execute Computational Workflows [40.572754656757475]
  This paper explores how these emerging capabilities can be harnessed to facilitate complex scientific research. We present initial findings from our attempt to integrate Phyloflow with OpenAI's function-calling API, and outline a strategy for developing a comprehensive workflow management system.
  arXiv  Detail & Related papers  (2023-12-12T20:17:13Z)
• Toward Automatically Completing GitHub Workflows [16.302521048148748]
  We present GH-WCOM (GitHub COMpletion), a Transformer-based approach supporting developers in writing a specific type of CI/CD pipelines, namely GitHub. Our empirical study shows that GH-WCOM provides up to 34.23% correct predictions.
  arXiv  Detail & Related papers  (2023-08-31T14:53:00Z)
• RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark [69.19502244910632]
  Combinatorial optimization (CO) is fundamental to several real-world applications, from logistics and scheduling to hardware design and resource allocation.<n>Deep reinforcement learning has recently shown significant benefits in solving CO problems, reducing reliance on domain expertise and improving computational efficiency.<n>We introduce RL4CO, a unified benchmark with in-depth library coverage of 27 CO problem environments and 23 state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-06-29T16:57:22Z)
• The GitHub Development Workflow Automation Ecosystems [47.818229204130596]
  Large-scale software development has become a highly collaborative endeavour. This chapter explores the ecosystems of development bots and GitHub Actions. It provides an extensive survey of the state-of-the-art in this domain.
  arXiv  Detail & Related papers  (2023-05-08T15:24:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.