Characterizing Issue Management in Runtime Systems

• URL: http://arxiv.org/abs/2310.15971v1
• Date: Tue, 24 Oct 2023 16:12:52 GMT
• Title: Characterizing Issue Management in Runtime Systems
• Authors: Salma Begum Tamanna, Gias Uddin, Lan Xia and Longyu Zhang
• Abstract summary: We report an empirical study of around 118K issues from 34 runtime system repos in GitHub. We found that issues regarding enhancement, test failure and bug are mostly posted on runtime system repositories. 82.65% issues are tagged with labels while only 28.30% issues have designated assignees.
• Score: 0.38233569758620056
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Modern programming languages like Java require runtime systems to support the implementation and deployment of software applications in diverse computing platforms and operating systems. These runtime systems are normally developed in GitHub-hosted repositories based on close collaboration between large software companies (e.g., IBM, Microsoft) and OSS developers. However, despite their popularity and broad usage; to the best of our knowledge, these repositories have never been studied. We report an empirical study of around 118K issues from 34 runtime system repos in GitHub. We found that issues regarding enhancement, test failure and bug are mostly posted on runtime system repositories and solution related discussion are mostly present on issue discussion. 82.69% issues in the runtime system repositories have been resolved and 0.69% issues are ignored; median of issue close rate, ignore rate and addressing time in these repositories are 76.1%, 2.2% and 58 days respectively. 82.65% issues are tagged with labels while only 28.30% issues have designated assignees and 90.65% issues contain at least one comment; also presence of these features in an issue report can affect issue closure. Based on the findings, we offer six recommendat

Related papers

• Classifying Issues in Open-source GitHub Repositories [0.0]
  GitHub is the most widely used platform for software maintenance in the open-source community.<n>Developers report issues on GitHub from time to time while facing difficulties.<n>Most of the GitHub repositories do not maintain regular labeling for the issues.
  arXiv  Detail & Related papers  (2025-07-25T06:20:54Z)
• CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance [18.886738819470086]
  We introduce CodeAssistBench (CAB), the first benchmark framework for evaluating multi-turn programming assistance.<n>Unlike existing programming Q&A benchmarks, CAB automatically generates scalable datasets from question-related GitHub issues.<n>Using this framework, we constructed a test set of 3,286 real-world programming questions across 231 repositories.
  arXiv  Detail & Related papers  (2025-07-14T17:19:00Z)
• "I wasn't sure if this is indeed a security risk": Data-driven Understanding of Security Issue Reporting in GitHub Repositories of Open Source npm Packages [8.360992461585308]
  This work collected 10,907,467 issues reported across GitHub repositories of 45,466 diverse npm packages.<n>We found that the tags associated with these issues indicate the existence of only 0.13% security-related issues.<n>Our approach of manual analysis followed by developing high accuracy machine learning models identify 1,617,738 security-related issues which are not tagged as security-related.
  arXiv  Detail & Related papers  (2025-06-09T13:11:35Z)
• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• CrashFixer: A crash resolution agent for the Linux kernel [58.152358195983155]
  This work builds upon kGym, which shares a benchmark for system-level Linux kernel bugs and a platform to run experiments on the Linux kernel. This paper introduces CrashFixer, the first LLM-based software repair agent that is applicable to Linux kernel bugs.
  arXiv  Detail & Related papers  (2025-04-29T04:18:51Z)
• Unveiling Pitfalls: Understanding Why AI-driven Code Agents Fail at GitHub Issue Resolution [22.03052751722933]
  Python execution errors during the issue resolution phase correlate with lower resolution rates and increased reasoning overheads. We have identified the most prevalent errors -- such as ModuleNotFoundError and TypeError -- and highlighted particularly challenging errors like OSError and database-related issues.
  arXiv  Detail & Related papers  (2025-03-16T06:24:51Z)
• SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? [64.34184587727334]
  We propose SWE-bench Multimodal to evaluate systems on their ability to fix bugs in visual, user-facing JavaScript software. SWE-bench M features 617 task instances collected from 17 JavaScript libraries used for web interface design, diagramming, data visualization, syntax highlighting, and interactive mapping. Our analysis finds that top-performing SWE-bench systems struggle with SWE-bench M, revealing limitations in visual problem-solving and cross-language generalization.
  arXiv  Detail & Related papers  (2024-10-04T18:48:58Z)
• Visual Analysis of GitHub Issues to Gain Insights [2.9051263101214566]
  This paper presents a prototype web application that generates visualizations to offer insights into issue timelines. It focuses on the lifecycle of issues and depicts vital information to enhance users' understanding of development patterns.
  arXiv  Detail & Related papers  (2024-07-30T15:17:57Z)
• KGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution [59.20933707301566]
  Large Language Models (LLMs) are consistently improving at increasingly realistic software engineering (SE) tasks. In real-world software stacks, significant SE effort is spent developing foundational system software like the Linux kernel. To evaluate if ML models are useful while developing such large-scale systems-level software, we introduce kGym and kBench.
  arXiv  Detail & Related papers  (2024-07-02T21:44:22Z)
• How to Understand Whole Software Repository? [64.19431011897515]
  An excellent understanding of the whole repository will be the critical path to Automatic Software Engineering (ASE) We develop a novel method named RepoUnderstander by guiding agents to comprehensively understand the whole repositories. To better utilize the repository-level knowledge, we guide the agents to summarize, analyze, and plan.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)
• Analyzing the Accessibility of GitHub Repositories for PyPI and NPM Libraries [91.97201077607862]
  Industrial applications heavily rely on open-source software (OSS) libraries, which provide various benefits. To monitor the activities of such communities, a comprehensive list of repositories for the libraries of an ecosystem must be accessible. In this study, we analyze the accessibility of GitHub repositories for PyPI and NPM libraries.
  arXiv  Detail & Related papers  (2024-04-26T13:27:04Z)
• MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution [47.850418420195304]
  Large Language Models (LLMs) have shown promise in code generation but face difficulties in resolving GitHub issues. We propose a novel Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four agents customized for software evolution.
  arXiv  Detail & Related papers  (2024-03-26T17:57:57Z)
• An Empirical Study on Workflows and Security Policies in Popular GitHub Repositories [9.048328480295224]
  In open-source projects, anyone can contribute, so it is important to have an active continuous integration and continuous delivery (CI/CD) pipeline. Many of these projects are hosted on GitHub, where maintainers can create automated security policies. We measure the usage of GitHub and security policies in thousands of popular repositories based on the number of stars.
  arXiv  Detail & Related papers  (2023-05-25T14:52:23Z)
• GiveMeLabeledIssues: An Open Source Issue Recommendation System [9.312780130838952]
  Developers often struggle to navigate an Open Source Software (OSS) project's issue-tracking system and find a suitable task. This paper presents a tool that mines project repositories and labels issues based on the skills required to solve them. GiveMeLabeledIssues facilitates matching developers' skills to tasks, reducing the burden on project maintainers.
  arXiv  Detail & Related papers  (2023-03-23T16:39:31Z)
• Using Developer Discussions to Guide Fixing Bugs in Software [51.00904399653609]
  We propose using bug report discussions, which are available before the task is performed and are also naturally occurring, avoiding the need for additional information from developers. We demonstrate that various forms of natural language context derived from such discussions can aid bug-fixing, even leading to improved performance over using commit messages corresponding to the oracle bug-fixing commits.
  arXiv  Detail & Related papers  (2022-11-11T16:37:33Z)
• GitRank: A Framework to Rank GitHub Repositories [0.0]
  Open-source repositories provide wealth of information and are increasingly being used to build artificial intelligence (AI) based systems. In this hackathon, we utilize known code quality measures and GrimoireLab toolkit to implement a framework, named GitRank, to rank open-source repositories on three different criteria.
  arXiv  Detail & Related papers  (2022-05-04T23:42:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.