CGP-Tuning: Structure-Aware Soft Prompt Tuning for Code Vulnerability Detection

• URL: http://arxiv.org/abs/2501.04510v1
• Date: Wed, 08 Jan 2025 13:56:17 GMT
• Title: CGP-Tuning: Structure-Aware Soft Prompt Tuning for Code Vulnerability Detection
• Authors: Ruijun Feng, Hammond Pearce, Pietro Liguori, Yulei Sui,
• Abstract summary: This paper introduces a new code graph-enhanced, structure-aware soft prompt tuning method for vulnerability detection, referred to as CGP-Tuning. It employs innovative type-aware embeddings to capture the rich semantic information within code graphs, along with a novel and efficient cross-modal alignment module. Experimental results demonstrate that CGP-Tuning outperforms the best state-of-the-art method by an average of 3.5 percentage points in accuracy.
• Score: 15.013699967804987
• License:
• Abstract: Large language models (LLMs) have been proposed as powerful tools for detecting software vulnerabilities, where task-specific fine-tuning is typically employed to provide vulnerability-specific knowledge to the LLMs for this purpose. However, traditional full-parameter fine-tuning is inefficient for modern, complex LLMs, which contain billions of parameters. Soft prompt tuning has been suggested as a more efficient alternative for fine-tuning LLMs in general cases. However, pure soft prompt tuning treats source code as plain text, losing structural information inherent in source code. Meanwhile, graph-enhanced soft prompt tuning methods, which aim to address this issue, are unable to preserve the rich semantic information within code graphs, as they are primarily designed for general graph-related tasks and focus more on adjacency information. They also fail to ensure computational efficiency while accounting for graph-text interactions. This paper, therefore, introduces a new code graph-enhanced, structure-aware soft prompt tuning method for vulnerability detection, referred to as CGP-Tuning. It employs innovative type-aware embeddings to capture the rich semantic information within code graphs, along with a novel and efficient cross-modal alignment module that achieves linear computational cost while incorporating graph-text interactions. The proposed CGP-Tuning is evaluated on the latest DiverseVul dataset and the most recent open-source code LLMs, CodeLlama and CodeGemma. Experimental results demonstrate that CGP-Tuning outperforms the best state-of-the-art method by an average of 3.5 percentage points in accuracy, without compromising its vulnerability detection capabilities for long source code.

Related papers

• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [70.72097493954067]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems. While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited. We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [51.19110891434727]
  Large Language Models (LLMs) with pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. E-LLaGNN is a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph.
  arXiv  Detail & Related papers  (2024-07-20T22:09:42Z)
• GLARE: Low Light Image Enhancement via Generative Latent Feature based Codebook Retrieval [80.96706764868898]
  We present a new Low-light Image Enhancement (LLIE) network via Generative LAtent feature based codebook REtrieval (GLARE) We develop a generative Invertible Latent Normalizing Flow (I-LNF) module to align the LL feature distribution to NL latent representations, guaranteeing the correct code retrieval in the codebook. Experiments confirm the superior performance of GLARE on various benchmark datasets and real-world data.
  arXiv  Detail & Related papers  (2024-07-17T09:40:15Z)
• Vulnerability-Hunter: An Adaptive Feature Perception Attention Network for Smart Contract Vulnerabilities [4.487191851300675]
  We propose AFPNet, a novel vulnerability detection model equipped with a feature perception module that has dynamic weights for comprehensive scanning of the entire smart contract code. We conduct an evaluation of AFPNet in the several large-scale datasets with vulnerability labels.
  arXiv  Detail & Related papers  (2024-07-07T10:13:41Z)
• GNNavi: Navigating the Information Flow in Large Language Models by Graph Neural Network [49.91919718254597]
  Large Language Models (LLMs) exhibit strong In-Context Learning capabilities when prompts with demonstrations are used. Prompt-based fine-tuning proves to be an effective fine-tuning method in low-data scenarios, but high demands on computing resources limit its practicality. GNNavi employs a Graph Neural Network layer to precisely guide the aggregation and distribution of information flow during the processing of prompts.
  arXiv  Detail & Related papers  (2024-02-18T21:13:05Z)
• Feature Engineering-Based Detection of Buffer Overflow Vulnerability in Source Code Using Neural Networks [2.9266864570485827]
  vulnerability detection method based on neural network models that learn features extracted from source codes. We maintain the semantic and syntactic information using state of the art word embedding algorithms such as GloVe and fastText. We have proposed a neural network model that can overcome issues associated with traditional neural networks.
  arXiv  Detail & Related papers  (2023-06-01T01:44:49Z)
• Automated Vulnerability Detection in Source Code Using Quantum Natural Language Processing [0.0]
  C and C++ open source code are now available in order to create a large-scale, classical machine-learning and quantum machine-learning system for function-level vulnerability identification. We created an efficient and scalable vulnerability detection method based on a deep neural network model Long Short Term Memory (LSTM), and quantum machine learning model Long Short Term Memory (QLSTM) The QLSTM with semantic and syntactic features detects significantly accurate vulnerability and runs faster than its classical counterpart.
  arXiv  Detail & Related papers  (2023-03-13T23:27:42Z)
• Lightweight Projective Derivative Codes for Compressed Asynchronous Gradient Descent [6.055286666916789]
  This paper proposes a novel algorithm that encodes the partial derivatives themselves and furthermore optimize the codes by performing lossy compression on the derivative codewords. The utility of this application of coding theory is a geometrical consequence of the observed fact in optimization research that noise is tolerable, sometimes even helpful, in gradient descent based learning algorithms.
  arXiv  Detail & Related papers  (2022-01-31T04:08:53Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection [20.65271290295621]
  We propose ReGVD, a graph network-based model for vulnerability detection. In particular, ReGVD views a given source code as a flat sequence of tokens. We obtain the highest accuracy on the real-world benchmark dataset from CodeXGLUE for vulnerability detection.
  arXiv  Detail & Related papers  (2021-10-14T12:44:38Z)
• Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation [57.92972327649165]
  This work explores a deep learning approach to automatically learn the insecure patterns from code corpora. Because code naturally admits graph structures with parsing, we develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program.
  arXiv  Detail & Related papers  (2021-09-07T21:24:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.