Source Code Vulnerability Detection: Combining Code Language Models and Code Property Graphs

• URL: http://arxiv.org/abs/2404.14719v1
• Date: Tue, 23 Apr 2024 03:48:18 GMT
• Title: Source Code Vulnerability Detection: Combining Code Language Models and Code Property Graphs
• Authors: Ruitong Liu, Yanbin Wang, Haitao Xu, Bin Liu, Jianguo Sun, Zhenhao Guo, Wenrui Ma,
• Abstract summary: Vul-LMGNN is a unified model that combines pre-trained code language models with code property graphs. Vul-LMGNN constructs a code property graph that integrates various code attributes into a unified graph structure. To effectively retain dependency information among various attributes, we introduce a gated code Graph Neural Network.
• Score: 5.953617559607503
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Currently, deep learning successfully applies to code vulnerability detection by learning from code sequences or property graphs. However, sequence-based methods often overlook essential code attributes such as syntax, control flow, and data dependencies, whereas graph-based approaches might underestimate the semantics of code and face challenges in capturing long-distance contextual information. To address this gap, we propose Vul-LMGNN, a unified model that combines pre-trained code language models with code property graphs for code vulnerability detection. Vul-LMGNN constructs a code property graph that integrates various code attributes (including syntax, flow control, and data dependencies) into a unified graph structure, thereafter leveraging pre-trained code model to extract local semantic features as node embeddings in the code property graph. Furthermore, to effectively retain dependency information among various attributes, we introduce a gated code Graph Neural Network (GNN). By jointly training the code language model and the gated code GNN modules in Vul-LMGNN, our proposed method efficiently leverages the strengths of both mechanisms. Finally, we utilize a pre-trained CodeBERT as an auxiliary classifier, with the final detection results derived by learning the linear interpolation of Vul-LMGNN and CodeBERT. The proposed method, evaluated across four real-world vulnerability datasets, demonstrated superior performance compared to six state-of-the-art approaches. Our source code could be accessed via the link: https://github.com/Vul-LMGNN/vul-LMGGNN.

Related papers

• Detecting Code Vulnerabilities with Heterogeneous GNN Training [3.1333320740278627]
  Graph Neural Network (GNN) machine learning can be a promising approach by modeling source code as graphs. This paper presents Inter-Procedural Abstract Graphs (IPAGs) as an efficient, language-agnostic representation of source code. We also propose a Heterogeneous Attention GNN (HAGNN) model that incorporates multiple subgraphs capturing different features of source code.
  arXiv  Detail & Related papers  (2025-02-24T04:39:16Z)
• GL-Fusion: Rethinking the Combination of Graph Neural Network and Large Language model [63.774726052837266]
  We introduce a new architecture that deeply integrates Graph Neural Networks (GNNs) with Large Language Models (LLMs) We introduce three key innovations: (1) Structure-Aware Transformers, which incorporate GNN's message-passing capabilities directly into LLM's transformer layers; (2) Graph-Text Cross-Attention, which processes full, uncompressed text from graph nodes and edges; and (3) GNN-LLM Twin Predictor, enabling LLM's flexible autoregressive generation alongside GNN's scalable one-pass prediction.
  arXiv  Detail & Related papers  (2024-12-08T05:49:58Z)
• Can Large Language Models Act as Ensembler for Multi-GNNs? [6.387816922598151]
  Graph Neural Networks (GNNs) have emerged as powerful models for learning from graph-structured data. GNNs lack the inherent semantic understanding capability of rich textual node attributes, limiting their effectiveness in applications. This research advances text-attributed graph ensemble learning by providing a robust and superior solution for integrating semantic and structural information.
  arXiv  Detail & Related papers  (2024-10-22T08:48:52Z)
• IDEA: A Flexible Framework of Certified Unlearning for Graph Neural Networks [68.6374698896505]
  Graph Neural Networks (GNNs) have been increasingly deployed in a plethora of applications. Privacy leakage may happen when the trained GNNs are deployed and exposed to potential attackers. We propose a principled framework named IDEA to achieve flexible and certified unlearning for GNNs.
  arXiv  Detail & Related papers  (2024-07-28T04:59:59Z)
• All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [51.19110891434727]
  Large Language Models (LLMs) with pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. E-LLaGNN is a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph.
  arXiv  Detail & Related papers  (2024-07-20T22:09:42Z)
• LOGIN: A Large Language Model Consulted Graph Neural Network Training Framework [30.54068909225463]
  We aim to streamline the GNN design process and leverage the advantages of Large Language Models (LLMs) to improve the performance of GNNs on downstream tasks. We formulate a new paradigm, coined "LLMs-as-Consultants," which integrates LLMs with GNNs in an interactive manner. We empirically evaluate the effectiveness of LOGIN on node classification tasks across both homophilic and heterophilic graphs.
  arXiv  Detail & Related papers  (2024-05-22T18:17:20Z)
• CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation [58.84212778960507]
  We propose CodeGRAG, a Graphical Retrieval Augmented Code Generation framework to enhance the performance of LLMs. CodeGRAG builds the graphical view of code blocks based on the control flow and data flow of them to fill the gap between programming languages and natural language. Various experiments and ablations are done on four datasets including both the C++ and python languages to validate the hard meta-graph prompt, the soft prompting technique, and the effectiveness of the objectives for pretrained GNN expert.
  arXiv  Detail & Related papers  (2024-05-03T02:48:55Z)
• Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs [59.74814230246034]
  Large Language Models (LLMs) have been proven to possess extensive common knowledge and powerful semantic comprehension abilities. We investigate two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors.
  arXiv  Detail & Related papers  (2023-07-07T05:31:31Z)
• Sequential Graph Neural Networks for Source Code Vulnerability Identification [5.582101184758527]
  We present a properly curated C/C++ source code vulnerability dataset to aid in developing models. We also propose a learning framework based on graph neural networks, denoted SEquential Graph Neural Network (SEGNN) for learning a large number of code semantic representations. Our evaluations on two datasets and four baseline methods in a graph classification setting demonstrate state-of-the-art results.
  arXiv  Detail & Related papers  (2023-05-23T17:25:51Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• GNN-LM: Language Modeling based on Global Contexts via GNN [32.52117529283929]
  We introduce GNN-LM, which extends the vanilla neural language model (LM) by allowing to reference similar contexts in the entire training corpus. GNN-LM achieves a new state-of-the-art perplexity of 14.8 on WikiText-103.
  arXiv  Detail & Related papers  (2021-10-17T07:18:21Z)
• ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection [20.65271290295621]
  We propose ReGVD, a graph network-based model for vulnerability detection. In particular, ReGVD views a given source code as a flat sequence of tokens. We obtain the highest accuracy on the real-world benchmark dataset from CodeXGLUE for vulnerability detection.
  arXiv  Detail & Related papers  (2021-10-14T12:44:38Z)
• Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation [57.92972327649165]
  This work explores a deep learning approach to automatically learn the insecure patterns from code corpora. Because code naturally admits graph structures with parsing, we develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program.
  arXiv  Detail & Related papers  (2021-09-07T21:24:36Z)
• AdaGNN: A multi-modal latent representation meta-learner for GNNs based on AdaBoosting [0.38073142980733]
  Graph Neural Networks (GNNs) focus on extracting intrinsic network features. We propose boosting-based meta learner for GNNs. AdaGNN performs exceptionally well for applications with rich and diverse node neighborhood information.
  arXiv  Detail & Related papers  (2021-08-14T03:07:26Z)
• deGraphCS: Embedding Variable-based Flow Graph for Neural Code Search [15.19181807445119]
  We propose a learnable deep Graph for Code Search (called deGraphCS) to transfer source code into variable-based flow graphs. We collect a large-scale dataset from GitHub containing 41,152 code snippets written in C language.
  arXiv  Detail & Related papers  (2021-03-24T06:57:44Z)
• Overcoming Catastrophic Forgetting in Graph Neural Networks [50.900153089330175]
  Catastrophic forgetting refers to the tendency that a neural network "forgets" the previous learned knowledge upon learning new tasks. We propose a novel scheme dedicated to overcoming this problem and hence strengthen continual learning in graph neural networks (GNNs) At the heart of our approach is a generic module, termed as topology-aware weight preserving(TWP)
  arXiv  Detail & Related papers  (2020-12-10T22:30:25Z)
• GraphCodeBERT: Pre-training Code Representations with Data Flow [97.00641522327699]
  We present GraphCodeBERT, a pre-trained model for programming language that considers the inherent structure of code. We use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables. We evaluate our model on four tasks, including code search, clone detection, code translation, and code refinement.
  arXiv  Detail & Related papers  (2020-09-17T15:25:56Z)
• Learning to map source code to software vulnerability using code-as-a-graph [67.62847721118142]
  We explore the applicability of Graph Neural Networks in learning the nuances of source code from a security perspective. We show that a code-as-graph encoding is more meaningful for vulnerability detection than existing code-as-photo and linear sequence encoding approaches.
  arXiv  Detail & Related papers  (2020-06-15T16:05:27Z)
• Improved Code Summarization via a Graph Neural Network [96.03715569092523]
  In general, source code summarization techniques use the source code as input and outputs a natural language description. We present an approach that uses a graph-based neural architecture that better matches the default structure of the AST to generate these summaries.
  arXiv  Detail & Related papers  (2020-04-06T17:36:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.