CppPerf: An Automated Pipeline and Dataset for Performance-Improving C++ Commits

• URL: http://arxiv.org/abs/2605.10890v1
• Date: Mon, 11 May 2026 17:33:32 GMT
• Title: CppPerf: An Automated Pipeline and Dataset for Performance-Improving C++ Commits
• Abstract summary: CppPerf-Mine is a pipeline that mines execution-time-improving patches from open-source C++ repositories on GitHub.<n>Using CppPerf-Mine, we build CppPerf-DB, a benchmark comprising 347 manually verified patches from 42 mature C++ repositories.<n>OpenHands correctly fixes only 13.5% of the patches in CppPerf-DB.
• Score: 6.283140995975512
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Recent progress in automated repair of performance bugs demands realistic, executable benchmarks. However, existing C++ performance benchmarks are largely built from competitive programming submissions, and recent real-world benchmarks predominantly target Python and .NET. To fill this gap, we present CppPerf-Mine, a configurable pipeline that mines execution-time-improving patches from open-source C++ repositories on GitHub by combining structural commit filtering, an LLM-based commit classifier, and a containerized build & test stage that produces fully reproducible Docker images for each patch. Using CppPerf-Mine, we build CppPerf-DB, a benchmark comprising 347 manually verified patches from 42 mature C++ repositories, 39% of which are multi-file, enabling the evaluation of repository-level repair tools. In our preliminary study, OpenHands correctly fixes only 13.5% of the patches in CppPerf-DB, confirming that real-world C++ performance repair remains an open challenge. CppPerf-Mine and CppPerf-DB are open-source and publicly available at: https://doi.org/10.5281/zenodo.20097425. In addition, a demonstration video is available at: https://www.youtube.com/watch?v=nixlupIgSdM.

Related papers

• Taming the Drift: Context-aware Repair of Dockerfile Drift during Software Evolution [52.50730821321986]
  Cadre is a context-aware framework for automated Dockerfile drift repair.<n>CDG maps each Dockerfile instruction to its file-level and inter-instruction dependencies.<n>DodeX mines real-world Dockerfile drift instances from GitHub Actions CI logs.
  arXiv  Detail & Related papers  (2026-07-14T09:12:26Z)
• PhantomRun: Auto Repair of Compilation Errors in Embedded Open Source Software [2.64399132991614]
  We study four major open-source embedded system projects, spanning over 4000 build failures from the project's CI runs.<n>We find that hardware dependencies account for the majority of compilation failures, followed by syntax errors and build-script issues.<n>We present PhantomRun, an automated framework that leverages large language models (LLMs) to generate and validate fixes for CI compilation failures.
  arXiv  Detail & Related papers  (2026-02-23T19:13:22Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• VibeTensor: System Software for Deep Learning, Fully Generated by AI Agents [42.56489784841984]
  "fully generated" refers to code provenance: implementation changes were produced and applied as agent-proposed diffs.<n>We describe the architecture, summarize the workflow used to produce and validate the system, and evaluate the artifact.
  arXiv  Detail & Related papers  (2026-01-21T19:29:00Z)
• RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository [52.98970048197381]
  RepoGenesis is the first multilingual benchmark for repository-level end-to-end web microservice generation.<n>It consists of 106 repositories (60 Python, 46 Java) across 18 domains and 11 frameworks, with 1,258 API endpoints and 2,335 test cases verified.<n>Results reveal that despite high AC (up to 73.91%) and DSR (up to 100%), the best-performing system achieves only 23.67% Pass@1 on Python and 21.45% on Java.
  arXiv  Detail & Related papers  (2026-01-20T13:19:20Z)
• Defects4C: Benchmarking Large Language Model Repair Capability with C/C++ Bugs [22.51003705398642]
  Automated Program Repair (APR) plays a critical role in enhancing the quality and reliability of software systems.<n>There remains a significant gap in research on C/C++ program repair, despite the widespread use of C/C++ and the prevalence of associated vulnerabilities.<n>We introduce Defects4C, a benchmark specifically designed for C/C++ program repair.
  arXiv  Detail & Related papers  (2025-10-13T06:49:28Z)
• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• CrashFixer: A crash resolution agent for the Linux kernel [58.152358195983155]
  This work builds upon kGym, which shares a benchmark for system-level Linux kernel bugs and a platform to run experiments on the Linux kernel.<n>This paper introduces CrashFixer, the first LLM-based software repair agent that is applicable to Linux kernel bugs.
  arXiv  Detail & Related papers  (2025-04-29T04:18:51Z)
• KGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution [59.20933707301566]
  Large Language Models (LLMs) are consistently improving at increasingly realistic software engineering (SE) tasks. In real-world software stacks, significant SE effort is spent developing foundational system software like the Linux kernel. To evaluate if ML models are useful while developing such large-scale systems-level software, we introduce kGym and kBench.
  arXiv  Detail & Related papers  (2024-07-02T21:44:22Z)
• PerfCurator: Curating a large-scale dataset of performance bug-related commits from public repositories [1.382112536681413]
  We propose PerfCurator, a repository miner that collects performance bug-related commits at scale. PerfCurator employs PcBERT-KD, a 125M parameter BERT model trained to classify performance bug-related commits. Our results demonstrate that this large-scale dataset significantly enhances the effectiveness of data-driven performance bug detection systems.
  arXiv  Detail & Related papers  (2024-06-17T16:49:13Z)
• Detecting Build Dependency Errors in Incremental Builds [13.823208277774572]
  We propose EChecker to detect build dependency errors in the context of incremental builds. EChecker automatically updates actual build dependencies by inferring them from C/C++ pre-processor directives and Makefile changes from new commits. EChecker increases the build dependency error detection efficiency by an average of 85.14 times.
  arXiv  Detail & Related papers  (2024-04-20T07:01:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.