Boidae: Your Personal Mining Platform

• URL: http://arxiv.org/abs/2401.11092v1
• Date: Sat, 20 Jan 2024 02:44:47 GMT
• Title: Boidae: Your Personal Mining Platform
• Abstract summary: We present Boidae, a family of Boa installations controlled and customized by users. In particular, Boidae allows the creation of custom datasets generated from any set of Git repositories. Boidae's scripts and all infrastructure it builds upon are open-sourced.
• Score: 0.21485350418225244
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Mining software repositories is a useful technique for researchers and practitioners to see what software developers actually do when developing software. Tools like Boa provide users with the ability to easily mine these open-source software repositories at a very large scale, with datasets containing hundreds of thousands of projects. The trade-off is that users must use the provided infrastructure, query language, runtime, and datasets and this might not fit all analysis needs. In this work, we present Boidae: a family of Boa installations controlled and customized by users. Boidae uses automation tools such as Ansible and Docker to facilitate the deployment of a customized Boa installation. In particular, Boidae allows the creation of custom datasets generated from any set of Git repositories, with helper scripts to aid in finding and cloning repositories from GitHub and SourceForge. In this paper, we briefly describe the architecture of Boidae and how researchers can utilize the infrastructure to generate custom datasets. Boidae's scripts and all infrastructure it builds upon are open-sourced. A video demonstration of Boidae's installation and extension is available at https://go.unl.edu/boidae.

Related papers

• A Dataset of Agentic AI Coding Tool Configurations [11.087963504113937]
  dataset covers 4,738 repositories across five tools (Claude Code, GitHub Copilot, OpenAI Codex, Cursor, Gemini) and eight configuration mechanisms.<n>We collected 15,591 configuration artifacts, the full content of 18,167 configuration files associated with these configuration artifacts, and 148,519 AI-co-authored commits.<n>This data supports research on context engineering, AI tool adoption patterns, and human-AI collaboration.
  arXiv  Detail & Related papers  (2026-05-08T19:58:27Z)
• PIPer: On-Device Environment Setup via Online Reinforcement Learning [74.52354321028493]
  Automated environment setup methods could assist developers by providing fully configured environments for arbitrary repositories without manual effort.<n>Recent studies reveal that even state-of-the-art Large Language Models (LLMs) achieve limited success in automating this task.<n>We combine supervised fine-tuning for generating correct scripts and Reinforcement Learning with Verifiable Rewards (RLVR) to adapt it to the task of environment setup.<n>On EnvBench-Python, our method enables Qwen3-8B (a model runnable on consumer hardware) to perform on par with larger models-Qwen3-32B and GPT-4
  arXiv  Detail & Related papers  (2025-09-29T20:03:05Z)
• GitHub Proxy Server: A tool for supporting massive data collection on GitHub [0.0]
  GitHub is the most popular social coding platform and widely used by developers and organizations to host their open-source projects around the world.<n>The platform has a web API that allow developers collect information from public repositories hosted on it.<n>However, collecting massive amount of data from GitHub can be very challenging due to existing restrictions and abuse detection mechanisms.<n>We present a tool, called GitHub Proxy Server, which abstracts such complexities into a tool that is independent on operational system and programming language.
  arXiv  Detail & Related papers  (2025-05-23T19:00:32Z)
• EnvBench: A Benchmark for Automated Environment Setup [76.02998475135824]
  Large Language Models have enabled researchers to focus on practical repository-level tasks in software engineering domain. Existing studies on environment setup introduce innovative agentic strategies, but their evaluation is often based on small datasets. To address this gap, we introduce a comprehensive environment setup benchmark EnvBench.
  arXiv  Detail & Related papers  (2025-03-18T17:19:12Z)
• PyPackIT: Automated Research Software Engineering for Scientific Python Applications on GitHub [0.0]
  PyPackIT is a user-friendly, ready-to-use software that enables scientists to focus on the scientific aspects of their projects. PyPackIT offers a robust project infrastructure including a build-ready Python package skeleton, a fully operational documentation and test suite, and a control center for dynamic project management.
  arXiv  Detail & Related papers  (2025-03-06T19:41:55Z)
• An Empirical Study of Dotfiles Repositories Containing User-Specific Configuration Files [1.7556600627464058]
  Hundreds of thousands choose to publicly host their repositories on GitHub. We collected and analyzed publicly-hosted dotfiles repositories on GitHub. We found that 25.8% of the top 500 most-starred GitHub users maintain some form of publicly accessible dotfiles repository.
  arXiv  Detail & Related papers  (2025-01-30T18:32:46Z)
• Cuvis.Ai: An Open-Source, Low-Code Software Ecosystem for Hyperspectral Processing and Classification [0.4038539043067986]
  cuvis.ai is an open-source and low-code software ecosystem for data acquisition, preprocessing, and model training. The package is written in Python and provides wrappers around common machine learning libraries.
  arXiv  Detail & Related papers  (2024-11-18T06:33:40Z)
• RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph [63.87660059104077]
  We present RepoGraph, a plug-in module that manages a repository-level structure for modern AI software engineering solutions. RepoGraph substantially boosts the performance of all systems, leading to a new state-of-the-art among open-source frameworks.
  arXiv  Detail & Related papers  (2024-10-03T05:45:26Z)
• How to Understand Whole Software Repository? [64.19431011897515]
  An excellent understanding of the whole repository will be the critical path to Automatic Software Engineering (ASE) We develop a novel method named RepoUnderstander by guiding agents to comprehensively understand the whole repositories. To better utilize the repository-level knowledge, we guide the agents to summarize, analyze, and plan.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)
• Chronicles of CI/CD: A Deep Dive into its Usage Over Time [0.5705775078773656]
  This paper analyzes the technologies developers use for CI/CD by analyzing GitHub repositories. Using a list of the state-of-the-art CI/CD technologies, we use the GitHub search API to find repositories using each of these technologies. We provide an overview of the use of CI/CD technologies in our days, but also what happened in the last 12 years.
  arXiv  Detail & Related papers  (2024-02-27T15:20:11Z)
• Testing GitHub projects on custom resources using unprivileged Kubernetes runners [1.137903861863692]
  GitHub is a popular repository for hosting software projects. Native GitHub Actions make it easy for software developers to validate new commits and have confidence that new code does not introduce major bugs. The freely available test environments are limited to only a few popular setups but can be extended with custom Action Runners.
  arXiv  Detail & Related papers  (2023-05-17T16:31:41Z)
• NVIDIA FLARE: Federated Learning from Simulation to Real-World [11.490933081543787]
  We created NVIDIA FLARE as an open-source development kit (SDK) to make it easier for data scientists to use FL in their research and real-world applications. The SDK includes solutions for state-of-the-art FL algorithms and federated machine learning approaches.
  arXiv  Detail & Related papers  (2022-10-24T14:30:50Z)
• DADApy: Distance-based Analysis of DAta-manifolds in Python [51.37841707191944]
  DADApy is a python software package for analysing and characterising high-dimensional data. It provides methods for estimating the intrinsic dimension and the probability density, for performing density-based clustering and for comparing different distance metrics.
  arXiv  Detail & Related papers  (2022-05-04T08:41:59Z)
• Repro: An Open-Source Library for Improving the Reproducibility and Usability of Publicly Available Research Code [74.28810048824519]
  Repro is an open-source library which aims at improving the usability of research code. It provides a lightweight Python API for running software released by researchers within Docker containers.
  arXiv  Detail & Related papers  (2022-04-29T01:54:54Z)
• DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population [95.0099875111663]
  DeepKE implements various information extraction tasks, including named entity recognition, relation extraction and attribute extraction. DeepKE allows developers and researchers to customize datasets and models to extract information from unstructured data according to their requirements.
  arXiv  Detail & Related papers  (2022-01-10T13:29:05Z)
• Datasets: A Community Library for Natural Language Processing [55.48866401721244]
  datasets is a community library for contemporary NLP. The library includes more than 650 unique datasets, has more than 250 contributors, and has helped support a variety of novel cross-dataset research projects.
  arXiv  Detail & Related papers  (2021-09-07T03:59:22Z)
• LabelGit: A Dataset for Software Repositories Classification using Attributed Dependency Graphs [11.523471275501857]
  We create a new dataset of GitHub projects called LabelGit. Our dataset uses direct information from the source code, like the dependency graph and source code neural representations from the identifiers. We hope to aid the development of solutions that do not rely on proxies but use the entire source code to perform classification.
  arXiv  Detail & Related papers  (2021-03-16T07:28:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.