PoCEvolve: Generating Proof-of-Concept Exploits from Security Patches with Vulnerability-Aware Prompt Evolution

• URL: http://arxiv.org/abs/2607.22076v1
• Date: Fri, 24 Jul 2026 08:20:28 GMT
• Title: PoCEvolve: Generating Proof-of-Concept Exploits from Security Patches with Vulnerability-Aware Prompt Evolution
• Abstract summary: We introduce PoCEvolve, a vulnerability-aware prompt-evolution framework that generates PoCs directly from vulnerability-fixing commits.<n>PoCEvolve assesses the usefulness of different dimensions of vulnerability-related context, including the inferred vulnerable API and code-coverage information.<n>When detailed vulnerability reports are available, PoCEvolve achieves a success rate of 71.7%, improving over PoCGen by 11.1%.
• Score: 9.074579150200455
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ideally, the detailed information about a vulnerability should be made available together with the fixing commit. In practice, however, such details often become available only long after the commit, even when a CVE has already been published. During this window, the patch is already public, so attackers can reverse-engineer it, yet defenders lack the details needed to assess exposure, prioritize, and validate the fix. Executable evidence, such as a proof-of-concept (PoC) exploit, could fill this gap. Prior work has automated PoC generation, but the state-of-the-art approach, PoCGen, assumes that a detailed vulnerability report is already available, which is precisely what is missing during this window. In this paper, we first present an empirical study quantifying the long delay between the fixing commit and the availability of a detailed vulnerability report. We then introduce PoCEvolve, a vulnerability-aware prompt-evolution framework that generates PoCs directly from vulnerability-fixing commits. Given a vulnerability-fixing commit, PoCEvolve synthesizes a corresponding PoC exploit. To learn from unsuccessful generation attempts, PoCEvolve assesses the usefulness of different dimensions of vulnerability-related context, including the inferred vulnerable API and code-coverage information. These assessments guide prompt evolution towards more effective exploit-generation prompts. We evaluate PoCEvolve on SecBench.VFC.js, where PoCEvolve achieves a PoC generation success rate of 58.4%, corresponding to relative improvements of 20.7% over PoCGen and 200.0% over the LLM baseline with GPT-4o-mini. With a recent model, Qwen3.7-Plus, PoCEvolve achieves a higher success rate of 85.3%. When detailed vulnerability reports are available, PoCEvolve achieves a success rate of 71.7%, improving over PoCGen by 11.1%.

Related papers

• Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale [54.83971397981572]
  Language-model agents increasingly operate over complete software repositories.<n> cybersecurity evaluations measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant code.<n>We study vulnerability localization: given a weakness class and an unfamiliar repository, identify the implementation files associated with that weakness.
  arXiv  Detail & Related papers  (2026-09-14T17:44:51Z)
• V2E: Validating Smart Contract Vulnerabilities through Profit-driven Exploit Generation and Execution [52.57923394143007]
  V2E is a new framework for validating whether a reported vulnerability is truly exploitable.<n>V2E automatically generates Proof-of-Concept (PoC) and then assess if the vulnerability could be triggered and incur any financial loss.<n> Evaluation on 264 manually labeled contracts shows that V2E outperforms the baseline approach.
  arXiv  Detail & Related papers  (2026-04-15T08:24:24Z)
• Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw [87.97230960702274]
  We present the first real-world safety evaluation of OpenClaw.<n>We introduce the CIK taxonomy, which unifies an agent's persistent state into three dimensions.<n>Our evaluations cover 12 attack scenarios on a live OpenClaw instance.
  arXiv  Detail & Related papers  (2026-04-06T15:27:05Z)
• SecPI: Secure Code Generation with Reasoning Models via Security Reasoning Internalization [50.71047638695205]
  Reasoning language models (RLMs) are increasingly used in programming.<n>Yet, even state-of-the-art RLMs frequently introduce critical security vulnerabilities in generated code.<n>We present SecPI, a fine-tuning pipeline that teaches RLMs to internalize structured security reasoning.
  arXiv  Detail & Related papers  (2026-04-04T04:29:11Z)
• CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability [50.57373283154859]
  We present CVE-Factory, the first multiagent framework to achieve expert-level quality in automatically transforming vulnerability tasks.<n>It is also evaluated on the latest realistic vulnerabilities and achieves a 66.2% verified success.<n>We synthesize over 1,000 executable training environments, the first large-scale scaling of agentic tasks in code security.
  arXiv  Detail & Related papers  (2026-02-03T02:27:16Z)
• Chasing One-day Vulnerabilities Across Open Source Forks [3.777973175977788]
  This paper presents a novel approach to help developers identify one-day vulnerabilities in forked repositories.<n>The approach propagates vulnerability information at the commit level and performs automated impact analysis.<n>It enables automatic detection of forked projects that have not incorporated fixes, leaving them potentially vulnerable.
  arXiv  Detail & Related papers  (2025-11-07T09:25:47Z)
• A Systematic Study on Generating Web Vulnerability Proof-of-Concepts Using Large Language Models [13.035038702785512]
  We present the first empirical study of Large Language Models (LLMs)-based PoC generation for web application vulnerabilities.<n>Our results show that LLMs can automatically generate working PoCs in 8%-34% of cases using only public data.<n>Further analysis shows that supplementing code context improves success rates by 17%-20%, with function-level providing 9%-13% improvement.
  arXiv  Detail & Related papers  (2025-10-11T10:15:38Z)
• What Do They Fix? LLM-Aided Categorization of Security Patches for Critical Memory Bugs [46.325755802511026]
  We developLM, a dual-method pipeline that integrates two approaches based on a Large Language Model (LLM) and a fine-tuned small language model.<n>LM successfully identified 111 of 5,140 recent Linux kernel patches addressing OOB or UAF vulnerabilities, with 90 true positives confirmed by manual verification.
  arXiv  Detail & Related papers  (2025-09-26T18:06:36Z)
• From Attack Descriptions to Vulnerabilities: A Sentence Transformer-Based Approach [0.39134914399411086]
  This paper evaluates 14 state-of-the-art sentence transformers for automatically identifying vulnerabilities from textual descriptions of attacks.<n>On average, 56% of the vulnerabilities identified by the MMPNet model are also represented within the CVE repository in conjunction with an attack.<n>A manual inspection of the results revealed the existence of 275 predicted links that were not documented in the MITRE repositories.
  arXiv  Detail & Related papers  (2025-09-02T08:27:36Z)
• PoCGen: Generating Proof-of-Concept Exploits for Vulnerabilities in Npm Packages [13.877936187495555]
  We present PoCGen, a novel approach to autonomously generate and validate PoC exploits for vulnerabilities in npm packages.<n>PoCGen successfully generates exploits for 77% of the vulnerabilities in the SecBench$.$js dataset.
  arXiv  Detail & Related papers  (2025-06-05T12:37:33Z)
• Unveiling Hidden Links Between Unseen Security Entities [3.7138962865789353]
  VulnScopper is an innovative approach that utilizes multi-modal representation learning, combining Knowledge Graphs (KG) and Natural Processing (NLP) We evaluate VulnScopper on two major security datasets, the National Vulnerability Database (NVD) and the Red Hat CVE database. Our results show that VulnScopper outperforms existing methods, achieving up to 78% Hits@10 accuracy in linking CVEs to Common Vulnerabilities and Exposures (CWEs), and Common Platform Languageions (CPEs)
  arXiv  Detail & Related papers  (2024-03-04T13:14:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.