Studying the Characteristics of AIOps Projects on GitHub

• URL: http://arxiv.org/abs/2212.13245v2
• Date: Tue, 5 Sep 2023 22:02:17 GMT
• Title: Studying the Characteristics of AIOps Projects on GitHub
• Authors: Roozbeh Aghili, Heng Li, Foutse Khomh
• Abstract summary: We conduct an in-depth analysis of open-source AIOps projects to understand the characteristics of AIOps in practice. We identify a set of AIOps projects from GitHub and analyze their repository metrics. Finally, we assess the quality of these projects using different quality metrics, such as the number of bugs.
• Score: 14.58848716249407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Artificial Intelligence for IT Operations (AIOps) leverages AI approaches to handle the massive amount of data generated during the operations of software systems. Prior works have proposed various AIOps solutions to support different tasks in system operations and maintenance, such as anomaly detection. In this study, we conduct an in-depth analysis of open-source AIOps projects to understand the characteristics of AIOps in practice. We first carefully identify a set of AIOps projects from GitHub and analyze their repository metrics (e.g., the used programming languages). Then, we qualitatively examine the projects to understand their input data, analysis techniques, and goals. Finally, we assess the quality of these projects using different quality metrics, such as the number of bugs. To provide context, we also sample two sets of baseline projects from GitHub: a random sample of machine learning projects and a random sample of general-purposed projects. By comparing different metrics between our identified AIOps projects and these baselines, we derive meaningful insights. Our results reveal a recent and growing interest in AIOps solutions. However, the quality metrics indicate that AIOps projects suffer from more issues than our baseline projects. We also pinpoint the most common issues in AIOps approaches and discuss potential solutions to address these challenges. Our findings offer valuable guidance to researchers and practitioners, enabling them to comprehend the current state of AIOps practices and shed light on different ways of improving AIOps' weaker aspects. To the best of our knowledge, this work marks the first attempt to characterize open-source AIOps projects.

Related papers

• Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition [24.845241768474363]
  LLM-powered tools like ChatGPT Data Analysis have the potential to help users tackle the challenging task of data analysis programming. However, our formative study uncovered serious challenges in verifying AI-generated results and steering the AI. We developed two contrasting approaches to address these challenges.
  arXiv  Detail & Related papers  (2024-07-02T20:33:50Z)
• A Survey of AIOps for Failure Management in the Era of Large Language Models [60.59720351854515]
  This paper presents a comprehensive survey of AIOps technology for failure management in the LLM era. It includes a detailed definition of AIOps tasks for failure management, the data sources for AIOps, and the LLM-based approaches adopted for AIOps.
  arXiv  Detail & Related papers  (2024-06-17T05:13:24Z)
• A Comprehensive Survey on Underwater Image Enhancement Based on Deep Learning [51.7818820745221]
  Underwater image enhancement (UIE) presents a significant challenge within computer vision research. Despite the development of numerous UIE algorithms, a thorough and systematic review is still absent.
  arXiv  Detail & Related papers  (2024-05-30T04:46:40Z)
• WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? [83.19032025950986]
  We study the use of large language model-based agents for interacting with software via web browsers. WorkArena is a benchmark of 33 tasks based on the widely-used ServiceNow platform. BrowserGym is an environment for the design and evaluation of such agents.
  arXiv  Detail & Related papers  (2024-03-12T14:58:45Z)
• PyRCA: A Library for Metric-based Root Cause Analysis [66.72542200701807]
  PyRCA is an open-source machine learning library of Root Cause Analysis (RCA) for Artificial Intelligence for IT Operations (AIOps) It provides a holistic framework to uncover the complicated metric causal dependencies and automatically locate root causes of incidents.
  arXiv  Detail & Related papers  (2023-06-20T09:55:10Z)
• AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges [60.56413461109281]
  Artificial Intelligence for IT operations (AIOps) aims to combine the power of AI with the big data generated by IT Operations processes. We discuss in depth the key types of data emitted by IT Operations activities, the scale and challenges in analyzing them, and where they can be helpful. We categorize the key AIOps tasks as - incident detection, failure prediction, root cause analysis and automated actions.
  arXiv  Detail & Related papers  (2023-04-10T15:38:12Z)
• OpenAGI: When LLM Meets Domain Experts [51.86179657467822]
  Human Intelligence (HI) excels at combining basic skills to solve complex tasks. This capability is vital for Artificial Intelligence (AI) and should be embedded in comprehensive AI Agents. We introduce OpenAGI, an open-source platform designed for solving multi-step, real-world tasks.
  arXiv  Detail & Related papers  (2023-04-10T03:55:35Z)
• Supporting the Task-driven Skill Identification in Open Source Project Issue Tracking Systems [0.0]
  We investigate the automatic labeling of open issues strategy to help the contributors to pick a task to contribute. By identifying the skills, we claim the contributor candidates should pick a task more suitable. We applied quantitative studies to analyze the relevance of the labels in an experiment and compare the strategies' relative importance.
  arXiv  Detail & Related papers  (2022-11-02T14:17:22Z)
• Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment [30.264524448340406]
  We formulate neural architecture search (NAS) tasks into general multi-objective optimization problems. We analyze the complex characteristics from an optimization point of view. We present an end-to-end pipeline, dubbed $texttEvoXBench$, to generate benchmark test problems for EMO algorithms to run efficiently.
  arXiv  Detail & Related papers  (2022-08-08T02:07:49Z)
• A Systematic Mapping Study in AIOps [0.6977626480948161]
  We conduct an in-depth mapping study to collect and organize the numerous scattered contributions to AIOps. We create an AIOps taxonomy to build a foundation for future contributions. Our results show a recent and growing interest towards AIOps, specifically to those contributions treating failure-related tasks.
  arXiv  Detail & Related papers  (2020-12-15T09:05:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.