CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training

• URL: http://arxiv.org/abs/2608.06471v1
• Date: Thu, 06 Aug 2026 18:03:27 GMT
• Title: CyberForge: Verified Vulnerability Injection at Repository Level for Cybersecurity Agent Training
• Abstract summary: CyberForge is a framework that synthesizes executable, repository-level security training data by injecting vulnerabilities into real C/C++ projects.<n>The resulting corpus holds 1034 validated vulnerabilities across 80 projects and 63 weakness categories.<n>Fine-tuning on trajectories collected over this corpus improves SEC-bench patch repair by +3.3 to +14.7 points.
• Score: 42.087410274052836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite recent advances, frontier large language model (LLM) agents remain limited in discovering and patching complex vulnerabilities in real-world software. Generally available agents can already aid attackers, who only need to find one exploitable weakness, while defenders must continuously identify and patch all vulnerabilities across fast-growing codebases. Stronger defensive agents would help close this gap, yet the scarcity of security training data with reproducible build and execution environments remains a bottleneck. We present CyberForge, a framework that synthesizes executable, repository-level security training data by injecting vulnerabilities into real C/C++ projects. It validates each instance dynamically: the injected build must pass the project's unit tests, and generated proof-of-vulnerability (PoV) must trigger on the injected build and not on the clean one. CyberForge is not limited by the availability of disclosed vulnerabilities, therefore it can scale in comparison to data augmentation techniques which rely on historic CVE data. The resulting corpus holds 1034 validated vulnerabilities across 80 projects and 63 weakness categories, with edit locality similar to real CVE patches under a real-versus-real noise floor. Fine-tuning on trajectories collected over this corpus improves SEC-bench patch repair by +3.3 to +14.7 points, in all six configurations of three model scales and two teachers, with the 31B student reaching its GPT-5.4-mini teacher, 72.7% against 74.0%. These gains generalize out of distribution to PatchEval, a corpus containing other programming languages, where every configuration also improves and the 31B student passes its teacher.

Related papers

• Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale [54.83971397981572]
  Language-model agents increasingly operate over complete software repositories.<n> cybersecurity evaluations measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant code.<n>We study vulnerability localization: given a weakness class and an unfamiliar repository, identify the implementation files associated with that weakness.
  arXiv  Detail & Related papers  (2026-09-14T17:44:51Z)
• CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild [49.44069582581181]
  CyberFactory transforms public vulnerability artifacts, including CVEs from the wild, into executable and verifiable task instances.<n>It uses a reusable vulnerability-analysis skill to guide the teacher through source inspection, problem solving with domain prior, and evidence-based validation.<n>On CyberGym, modelname reaches 52.4% Pass@1 under a one-hour budget, improving over its Qwen3.5 base model by +22.8 points.
  arXiv  Detail & Related papers  (2026-08-24T12:30:10Z)
• VICBench: A Multi-Language Benchmark for Code Vulnerability Detection [12.72412925135797]
  Existing vulnerability datasets suffer from limited programming language coverage, restricted patch complexity, and narrow project scope.<n>We create a benchmark - VICBench - of 100 verified VICs for 100 CVEs across 88 projects in Python, Java, and C++, covering 48 CWE types.<n> VICBench features complex real-world vulnerability fixes averaging 38.6 lines and corresponding VICs of 252.5 lines - significantly larger than prior work.
  arXiv  Detail & Related papers  (2026-08-12T16:45:49Z)
• PoCEvolve: Generating Proof-of-Concept Exploits from Security Patches with Vulnerability-Aware Prompt Evolution [9.074579150200455]
  We introduce PoCEvolve, a vulnerability-aware prompt-evolution framework that generates PoCs directly from vulnerability-fixing commits.<n>PoCEvolve assesses the usefulness of different dimensions of vulnerability-related context, including the inferred vulnerable API and code-coverage information.<n>When detailed vulnerability reports are available, PoCEvolve achieves a success rate of 71.7%, improving over PoCGen by 11.1%.
  arXiv  Detail & Related papers  (2026-07-24T08:20:28Z)
• Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens [65.53494487819053]
  Claw-like AI agents (e.g., OpenClaw) are always-on processes with persistent access to credentials, files, tools, and external services.<n>We adopt a computer-system analogy: treating a Claw-like agent as an agentic computer system whose gateway runtime plays an OS-like mediation role.<n>We develop SafeClawArena, a benchmark of 406 adversarial tasks across four attack surfaces.
  arXiv  Detail & Related papers  (2026-06-29T18:00:45Z)
• RAVEN: Agentic RAG for Automated Vulnerability Repair [48.8954115363348]
  We propose RAVEN, a scalable, efficient and autonomous framework for automated vulnerability repair.<n>RAVEN integrates an agentic retrieval-augmented generation (RAG) pipeline with controlled iterative repair in a unified framework.<n>RAVEN achieves an overall repair success rate of 83.13%, outperforming all existing state-of-the-art repair frameworks.
  arXiv  Detail & Related papers  (2026-06-21T19:18:06Z)
• Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw [87.97230960702274]
  We present the first real-world safety evaluation of OpenClaw.<n>We introduce the CIK taxonomy, which unifies an agent's persistent state into three dimensions.<n>Our evaluations cover 12 attack scenarios on a live OpenClaw instance.
  arXiv  Detail & Related papers  (2026-04-06T15:27:05Z)
• CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability [50.57373283154859]
  We present CVE-Factory, the first multiagent framework to achieve expert-level quality in automatically transforming vulnerability tasks.<n>It is also evaluated on the latest realistic vulnerabilities and achieves a 66.2% verified success.<n>We synthesize over 1,000 executable training environments, the first large-scale scaling of agentic tasks in code security.
  arXiv  Detail & Related papers  (2026-02-03T02:27:16Z)
• CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale [45.97598662617568]
  We introduce CyberGym, a large-scale benchmark featuring 1,507 real-world vulnerabilities across 188 software projects.<n>We show that CyberGym leads to the discovery of 35 zero-day vulnerabilities and 17 historically incomplete patches.<n>These results underscore that CyberGym is not only a robust benchmark for measuring AI's progress in cybersecurity but also a platform for creating direct, real-world security impact.
  arXiv  Detail & Related papers  (2025-06-03T07:35:14Z)
• There are More Fish in the Sea: Automated Vulnerability Repair via Binary Templates [4.907610470063863]
  We propose a template-based automated vulnerability repair approach for Java binaries.<n>Experiments on the Vul4J dataset demonstrate that TemVUR successfully repairs 11 vulnerabilities.<n>To assess the generalizability of TemVUR, we curate the ManyVuls4J dataset.
  arXiv  Detail & Related papers  (2024-11-27T06:59:45Z)
• Cybersecurity Defenses: Exploration of CVE Types through Attack Descriptions [1.0474508494260908]
  VULDAT is a classification tool using a sentence transformer MPNET to identify system vulnerabilities from attack descriptions. Our model was applied to 100 attack techniques from the ATT&CK repository and 685 issues from the CVE repository. Our findings indicate that our model achieves the best performance with F1 score of 0.85, Precision of 0.86, and Recall of 0.83.
  arXiv  Detail & Related papers  (2024-07-09T11:08:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.