VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model

• URL: http://arxiv.org/abs/2308.04662v3
• Date: Sat, 18 May 2024 13:49:35 GMT
• Title: VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model
• Authors: Tianyu Chen, Lin Li, Liuchuan Zhu, Zongyang Li, Xueqing Liu, Guangtai Liang, Qianxiang Wang, Tao Xie,
• Abstract summary: VulLibGen is a method to directly generate affected packages. It has an average accuracy of 0.806 for identifying vulnerable packages. We have submitted 60 vulnerability, affected package> pairs to GitHub Advisory.
• Score: 13.96251273677855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Security practitioners maintain vulnerability reports (e.g., GitHub Advisory) to help developers mitigate security risks. An important task for these databases is automatically extracting structured information mentioned in the report, e.g., the affected software packages, to accelerate the defense of the vulnerability ecosystem. However, it is challenging for existing work on affected package identification to achieve a high accuracy. One reason is that all existing work focuses on relatively smaller models, thus they cannot harness the knowledge and semantic capabilities of large language models. To address this limitation, we propose VulLibGen, the first method to use LLM for affected package identification. In contrast to existing work, VulLibGen proposes the novel idea to directly generate the affected package. To improve the accuracy, VulLibGen employs supervised fine-tuning (SFT), retrieval augmented generation (RAG) and a local search algorithm. The local search algorithm is a novel postprocessing algorithm we introduce for reducing the hallucination of the generated packages. Our evaluation results show that VulLibGen has an average accuracy of 0.806 for identifying vulnerable packages in the four most popular ecosystems in GitHub Advisory (Java, JS, Python, Go) while the best average accuracy in previous work is 0.721. Additionally, VulLibGen has high value to security practice: we submitted 60 <vulnerability, affected package> pairs to GitHub Advisory (covers four ecosystems). 34 of them have been accepted and merged and 20 are pending approval. Our code and dataset can be found in the attachments.

Related papers

• RedCode: Risky Code Execution and Generation Benchmark for Code Agents [50.81206098588923]
  RedCode is a benchmark for risky code execution and generation. RedCode-Exec provides challenging prompts that could lead to risky code execution. RedCode-Gen provides 160 prompts with function signatures and docstrings as input to assess whether code agents will follow instructions.
  arXiv  Detail & Related papers  (2024-11-12T13:30:06Z)
• "Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models [74.05368440735468]
  Retrieval-Augmented Generative (RAG) models enhance Large Language Models (LLMs) In this paper, we demonstrate a security threat where adversaries can exploit the openness of these knowledge bases.
  arXiv  Detail & Related papers  (2024-06-26T05:36:23Z)
• VulZoo: A Comprehensive Vulnerability Intelligence Dataset [12.229092589037808]
  VulZoo is a comprehensive vulnerability intelligence dataset that covers 17 popular vulnerability information sources. We make VulZoo publicly available and maintain it with incremental updates to facilitate future research.
  arXiv  Detail & Related papers  (2024-06-24T06:39:07Z)
• How to Understand Whole Software Repository? [64.19431011897515]
  An excellent understanding of the whole repository will be the critical path to Automatic Software Engineering (ASE) We develop a novel method named RepoUnderstander by guiding agents to comprehensively understand the whole repositories. To better utilize the repository-level knowledge, we guide the agents to summarize, analyze, and plan.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)
• Empirical Analysis of Vulnerabilities Life Cycle in Golang Ecosystem [0.773844059806915]
  A comprehensive investigation was undertaken to examine the life cycle of vulnerability in Golang. It turned out that 66.10% of modules in the Golang ecosystem were affected by vulnerabilities. By analyzing reasons behind non-lagged and lagged vulnerabilities, timely releasing and indexing patch versions could significantly enhance ecosystem security.
  arXiv  Detail & Related papers  (2023-12-31T14:53:51Z)
• On the Security Blind Spots of Software Composition Analysis [46.1389163921338]
  We present a novel approach to detect vulnerable clones in the Maven repository. We retrieve over 53k potential vulnerable clones from Maven Central. We detect 727 confirmed vulnerable clones and synthesize a testable proof-of-vulnerability project for each of those.
  arXiv  Detail & Related papers  (2023-06-08T20:14:46Z)
• Multi-Granularity Detector for Vulnerability Fixes [13.653249890867222]
  We propose MiDas (Multi-Granularity Detector for Vulnerability Fixes) to identify vulnerability-fixing commits. MiDas constructs different neural networks for each level of code change granularity, corresponding to commit-level, file-level, hunk-level, and line-level. MiDas outperforms the current state-of-the-art baseline in terms of AUC by 4.9% and 13.7% on Java and Python-based datasets.
  arXiv  Detail & Related papers  (2023-05-23T10:06:28Z)
• CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
  Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
  arXiv  Detail & Related papers  (2023-02-08T11:54:07Z)
• VulCurator: A Vulnerability-Fixing Commit Detector [8.32137934421055]
  VulCurator is a tool that leverages deep learning on richer sources of information. VulCurator outperforms the state-of-the-art baselines up to 16.1% in terms of F1-score.
  arXiv  Detail & Related papers  (2022-09-07T16:11:31Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• Automated Mapping of Vulnerability Advisories onto their Fix Commits in Open Source Repositories [7.629717457706326]
  We present an approach that combines practical experience and machine-learning (ML) An advisory record containing key information about a vulnerability is extracted from an advisory. A subset of candidate fix commits is obtained from the source code repository of the affected project.
  arXiv  Detail & Related papers  (2021-03-24T17:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.