PoCGen: Generating Proof-of-Concept Exploits for Vulnerabilities in Npm Packages

• URL: http://arxiv.org/abs/2506.04962v3
• Date: Fri, 26 Sep 2025 22:43:31 GMT
• Title: PoCGen: Generating Proof-of-Concept Exploits for Vulnerabilities in Npm Packages
• Authors: Deniz Simsek, Aryaz Eghbali, Michael Pradel,
• Abstract summary: We present PoCGen, a novel approach to autonomously generate and validate PoC exploits for vulnerabilities in npm packages.<n>PoCGen successfully generates exploits for 77% of the vulnerabilities in the SecBench$.$js dataset.
• Score: 13.877936187495555
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Security vulnerabilities in software packages are a significant concern for developers and users alike. Patching these vulnerabilities in a timely manner is crucial to restoring the integrity and security of software systems. However, previous work has shown that vulnerability reports often lack proof-of-concept (PoC) exploits, which are essential for fixing the vulnerability, testing patches, and avoiding regressions. Creating a PoC exploit is challenging because vulnerability reports are informal and often incomplete, and because it requires a detailed understanding of how inputs passed to potentially vulnerable APIs may reach security-relevant sinks. In this paper, we present PoCGen, a novel approach to autonomously generate and validate PoC exploits for vulnerabilities in npm packages. The approach is the first to address this task by combining the complementary strengths of large language models (LLMs), e.g., to understand informal vulnerability reports, with static analysis, e.g., to identify taint paths, and dynamic analysis, e.g., to validate generated exploits. PoCGen successfully generates exploits for 77% of the vulnerabilities in the SecBench$.$js dataset. This success rate significantly outperforms a recent baseline (by 45 absolute percentage points), while imposing an average cost of only $0.02 per generated exploit. Moreover, PoCGen generates six successful exploits for recent real-world vulnerabilities, five of which are now included in their respective vulnerability reports.

Related papers

• Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents [57.49020237126194]
  Large language models (LLMs) have shown promise in assisting cybersecurity tasks, yet existing approaches struggle with automatic vulnerability discovery and exploitation.<n>We propose Co-RedTeam, a security-aware multi-agent framework designed to mirror real-world red-teaming.<n>Co-RedTeam decomposes vulnerability analysis into coordinated discovery and exploitation stages, enabling agents to plan, execute, validate, and refine actions.
  arXiv  Detail & Related papers  (2026-02-02T14:38:45Z)
• Chasing One-day Vulnerabilities Across Open Source Forks [3.777973175977788]
  This paper presents a novel approach to help developers identify one-day vulnerabilities in forked repositories.<n>The approach propagates vulnerability information at the commit level and performs automated impact analysis.<n>It enables automatic detection of forked projects that have not incorporated fixes, leaving them potentially vulnerable.
  arXiv  Detail & Related papers  (2025-11-07T09:25:47Z)
• From Model to Breach: Towards Actionable LLM-Generated Vulnerabilities Reporting [43.57360781012506]
  We show that even the latest open-weight models are vulnerable in the earliest reported vulnerability scenarios.<n>We introduce a new severity metric that reflects the risk posed by an LLM-generated vulnerability.<n>To encourage the mitigation of the most serious and prevalent vulnerabilities, we use PE to define the Model Exposure (ME) score.
  arXiv  Detail & Related papers  (2025-11-06T16:52:27Z)
• What Do They Fix? LLM-Aided Categorization of Security Patches for Critical Memory Bugs [46.325755802511026]
  We developLM, a dual-method pipeline that integrates two approaches based on a Large Language Model (LLM) and a fine-tuned small language model.<n>LM successfully identified 111 of 5,140 recent Linux kernel patches addressing OOB or UAF vulnerabilities, with 90 true positives confirmed by manual verification.
  arXiv  Detail & Related papers  (2025-09-26T18:06:36Z)
• Weakly Supervised Vulnerability Localization via Multiple Instance Learning [46.980136742826836]
  We propose a novel approach called WAVES for WeAkly supervised Vulnerability localization via multiplE inStance learning.<n>WAVES has the capability to determine whether a function is vulnerable (i.e., vulnerability detection) and pinpoint the vulnerable statements.<n>Our approach achieves comparable performance in vulnerability detection and state-of-the-art performance in statement-level vulnerability localization.
  arXiv  Detail & Related papers  (2025-09-14T15:11:39Z)
• VulnRepairEval: An Exploit-Based Evaluation Framework for Assessing Large Language Model Vulnerability Repair Capabilities [41.85494398578654]
  VulnRepairEval is an evaluation framework anchored in functional Proof-of-Concept exploits.<n>Our framework delivers a comprehensive, containerized evaluation pipeline that enables reproducible differential assessment.
  arXiv  Detail & Related papers  (2025-09-03T14:06:10Z)
• From Attack Descriptions to Vulnerabilities: A Sentence Transformer-Based Approach [0.39134914399411086]
  This paper evaluates 14 state-of-the-art sentence transformers for automatically identifying vulnerabilities from textual descriptions of attacks.<n>On average, 56% of the vulnerabilities identified by the MMPNet model are also represented within the CVE repository in conjunction with an attack.<n>A manual inspection of the results revealed the existence of 275 predicted links that were not documented in the MITRE repositories.
  arXiv  Detail & Related papers  (2025-09-02T08:27:36Z)
• Decompiling Smart Contracts with a Large Language Model [51.49197239479266]
  Despite Etherscan's 78,047,845 smart contracts deployed on (as of May 26, 2025), a mere 767,520 ( 1%) are open source.<n>This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode.<n>We introduce a pioneering decompilation pipeline that transforms bytecode into human-readable and semantically faithful Solidity code.
  arXiv  Detail & Related papers  (2025-06-24T13:42:59Z)
• FORGE: An LLM-driven Framework for Large-Scale Smart Contract Vulnerability Dataset Construction [34.20628333535654]
  FORGE is the first automated approach for constructing smart contract vulnerability datasets.<n>We generate a dataset comprising 81,390 solidity files and 27,497 vulnerability findings across 296 CWE categories.<n>Results reveal the significant limitations in current detection capabilities.
  arXiv  Detail & Related papers  (2025-06-23T16:03:16Z)
• CyberGym: Evaluating AI Agents' Cybersecurity Capabilities with Real-World Vulnerabilities at Scale [46.76144797837242]
  Large language model (LLM) agents are becoming increasingly skilled at handling cybersecurity tasks autonomously.<n>Existing benchmarks fall short, often failing to capture real-world scenarios or being limited in scope.<n>We introduce CyberGym, a large-scale and high-quality cybersecurity evaluation framework featuring 1,507 real-world vulnerabilities.
  arXiv  Detail & Related papers  (2025-06-03T07:35:14Z)
• Eradicating the Unseen: Detecting, Exploiting, and Remediating a Path Traversal Vulnerability across GitHub [1.2124551005857036]
  Vulnerabilities in open-source software can cause cascading effects in the modern digital ecosystem.<n>We identified 1,756 vulnerable open-source projects, some of which are very influential.<n>We have responsibly disclosed the vulnerability to the maintainers, and 14% of the reported vulnerabilities have been remediated.
  arXiv  Detail & Related papers  (2025-05-26T16:29:21Z)
• The Ripple Effect of Vulnerabilities in Maven Central: Prevalence, Propagation, and Mitigation Challenges [8.955037553566774]
  We analyze the prevalence and impact of vulnerabilities within the Maven Central ecosystem using Common Vulnerabilities and Exposures data.<n>In our subsample of around 4 million releases, we found that while only about 1% of releases have direct vulnerabilities.<n>We also observed that the time taken to patch vulnerabilities, including those of high or critical severity, often spans several years.
  arXiv  Detail & Related papers  (2025-04-05T13:45:27Z)
• There are More Fish in the Sea: Automated Vulnerability Repair via Binary Templates [4.907610470063863]
  We propose a template-based automated vulnerability repair approach for Java binaries.<n>Experiments on the Vul4J dataset demonstrate that TemVUR successfully repairs 11 vulnerabilities.<n>To assess the generalizability of TemVUR, we curate the ManyVuls4J dataset.
  arXiv  Detail & Related papers  (2024-11-27T06:59:45Z)
• Measuring the Exploitation of Weaknesses in the Wild [0.0]
  A weakness is a bug or fault type that can be exploited through an operation that results in a security-relevant error. This work introduces a simple metric to determine the probability of a weakness being exploited in the wild for any 30-day window. Our analysis reveals that 92 % of the weaknesses are not being constantly exploited.
  arXiv  Detail & Related papers  (2024-05-02T13:49:51Z)
• Vulnerability Detection with Code Language Models: How Far Are We? [40.455600722638906]
  PrimeVul is a new dataset for training and evaluating code LMs for vulnerability detection. It incorporates a novel set of data labeling techniques that achieve comparable label accuracy to human-verified benchmarks. It also implements a rigorous data de-duplication and chronological data splitting strategy to mitigate data leakage issues.
  arXiv  Detail & Related papers  (2024-03-27T14:34:29Z)
• Profile of Vulnerability Remediations in Dependencies Using Graph Analysis [40.35284812745255]
  This research introduces graph analysis methods and a modified Graph Attention Convolutional Neural Network (GAT) model. We analyze control flow graphs to profile breaking changes in applications occurring from dependency upgrades intended to remediate vulnerabilities. Results demonstrate the effectiveness of the enhanced GAT model in offering nuanced insights into the relational dynamics of code vulnerabilities.
  arXiv  Detail & Related papers  (2024-03-08T02:01:47Z)
• Unveiling Hidden Links Between Unseen Security Entities [3.7138962865789353]
  VulnScopper is an innovative approach that utilizes multi-modal representation learning, combining Knowledge Graphs (KG) and Natural Processing (NLP) We evaluate VulnScopper on two major security datasets, the National Vulnerability Database (NVD) and the Red Hat CVE database. Our results show that VulnScopper outperforms existing methods, achieving up to 78% Hits@10 accuracy in linking CVEs to Common Vulnerabilities and Exposures (CWEs), and Common Platform Languageions (CPEs)
  arXiv  Detail & Related papers  (2024-03-04T13:14:39Z)
• SliceLocator: Locating Vulnerable Statements with Graph-based Detectors [33.395068754566935]
  SliceLocator identifies the most relevant taint flow by selecting the highest-weighted flow path from all potential vulnerability-triggering statements.<n>We demonstrate that SliceLocator consistently performs well on four state-of-the-art GNN-based vulnerability detectors.
  arXiv  Detail & Related papers  (2024-01-05T10:15:04Z)
• Exploiting Library Vulnerability via Migration Based Automating Test Generation [16.39796265296833]
  In software development, developers extensively utilize third-party libraries to avoid implementing existing functionalities. Vulnerability exploits, as code snippets provided for reproducing vulnerabilities after disclosure, contain a wealth of vulnerability-related information. This study proposes a new method based on vulnerability exploits, called VESTA, which provides vulnerability exploit tests as the basis for developers to decide whether to update dependencies.
  arXiv  Detail & Related papers  (2023-12-15T06:46:45Z)
• REEF: A Framework for Collecting Real-World Vulnerabilities and Fixes [40.401211102969356]
  We propose an automated collecting framework REEF to collect REal-world vulnErabilities and Fixes from open-source repositories. We develop a multi-language crawler to collect vulnerabilities and their fixes, and design metrics to filter for high-quality vulnerability-fix pairs. Through extensive experiments, we demonstrate that our approach can collect high-quality vulnerability-fix pairs and generate strong explanations.
  arXiv  Detail & Related papers  (2023-09-15T02:50:08Z)
• Enriching Vulnerability Reports Through Automated and Augmented Description Summarization [6.3455238301221675]
  Vulnerability descriptions play an important role in communicating the vulnerability information to security analysts. This paper devises a pipeline to augment vulnerability description through third party reference (hyperlink) scrapping.
  arXiv  Detail & Related papers  (2022-10-03T22:46:35Z)
• Certifiers Make Neural Networks Vulnerable to Availability Attacks [70.69104148250614]
  We show for the first time that fallback strategies can be deliberately triggered by an adversary. In addition to naturally occurring abstains for some inputs and perturbations, the adversary can use training-time attacks to deliberately trigger the fallback. We design two novel availability attacks, which show the practical relevance of these threats.
  arXiv  Detail & Related papers  (2021-08-25T15:49:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.