Efficient Software Vulnerability Detection Using Transformer-based Models

• URL: http://arxiv.org/abs/2604.00112v1
• Date: Tue, 31 Mar 2026 18:23:04 GMT
• Title: Efficient Software Vulnerability Detection Using Transformer-based Models
• Abstract summary: Deep neural networks have shown promising results on vulnerability detection, but they lack the capability to capture global contextual information on vulnerable code.<n>We use program slices that encapsulate key syntactic and semantic features of program code, such as API function calls, array usage, pointer manipulations, and arithmetic expressions.<n>Our work offers a robust and efficient approach to identifying vulnerable code with moderate resource usage and training time.
• Score: 4.786839591214602
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Detecting software vulnerabilities is critical to ensuring the security and reliability of modern computer systems. Deep neural networks have shown promising results on vulnerability detection, but they lack the capability to capture global contextual information on vulnerable code. To address this limitation, we explore the application of transformers for C/C++ vulnerability detection. We use program slices that encapsulate key syntactic and semantic features of program code, such as API function calls, array usage, pointer manipulations, and arithmetic expressions. By leveraging transformers' capability to capture both local and global contextual information on vulnerable code, our work can identify vulnerabilities accurately. Combined with data balancing and hyperparameter fine-tuning, our work offers a robust and efficient approach to identifying vulnerable code with moderate resource usage and training time.

Related papers

• CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilities [66.48570184296857]
  CyberGym-E2E is a large-scale and realistic end-to-end cybersecurity benchmark.<n>It comprehensively evaluates AI agents' abilities across the full lifecycle of vulnerability discovery, PoC generation, and patch generation.<n>The benchmark consists of 920 real-world vulnerabilities across 139 different open-source projects.
  arXiv  Detail & Related papers  (2026-06-03T05:06:37Z)
• SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization [61.91729298584227]
  We introduce SecureForge, an automated pipeline that audits security risks of frontier models and produces auditing-informed secure system prompts.<n>SecureForge first identifies benign prompts that produce statically detectable vulnerabilities, and then amplifies them into a large synthetic prompt corpus of diverse scenarios.<n>On frontier models, SecureForge yields a statistically significant improvement in both unit test success and output security, with output vulnerabilities reduced by up to 48%.
  arXiv  Detail & Related papers  (2026-05-08T18:40:47Z)
• From Theory to Practice: Code Generation Using LLMs for CAPEC and CWE Frameworks [2.7662026539843123]
  We present a novel dataset that provides examples of vulnerable code snippets corresponding to CAPEC and CWE descriptions.<n>This dataset is a valuable resource for training machine learning models focused on automatic vulnerability detection and remediation.<n>The final dataset comprises 615 CAPEC code snippets in three programming languages: Java, Python, and JavaScript.
  arXiv  Detail & Related papers  (2026-04-02T21:56:41Z)
• Distilling Lightweight Language Models for C/C++ Vulnerabilities [7.45549460594508]
  FineSec is a novel framework that harnesses Large Language Models through knowledge distillation to enable efficient and precise vulnerability identification in C/C++s.<n>By integrating data preparation, training, evaluation, and continuous learning into a unified, single-task workflow, FineSec offers a streamlined approach.
  arXiv  Detail & Related papers  (2025-10-08T04:58:51Z)
• Weakly Supervised Vulnerability Localization via Multiple Instance Learning [46.980136742826836]
  We propose a novel approach called WAVES for WeAkly supervised Vulnerability localization via multiplE inStance learning.<n>WAVES has the capability to determine whether a function is vulnerable (i.e., vulnerability detection) and pinpoint the vulnerable statements.<n>Our approach achieves comparable performance in vulnerability detection and state-of-the-art performance in statement-level vulnerability localization.
  arXiv  Detail & Related papers  (2025-09-14T15:11:39Z)
• CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale [45.97598662617568]
  We introduce CyberGym, a large-scale benchmark featuring 1,507 real-world vulnerabilities across 188 software projects.<n>We show that CyberGym leads to the discovery of 35 zero-day vulnerabilities and 17 historically incomplete patches.<n>These results underscore that CyberGym is not only a robust benchmark for measuring AI's progress in cybersecurity but also a platform for creating direct, real-world security impact.
  arXiv  Detail & Related papers  (2025-06-03T07:35:14Z)
• Enhancing Software Vulnerability Detection Using Code Property Graphs and Convolutional Neural Networks [0.0]
  This paper proposes a novel approach to detecting software vulnerabilities using a combination of code property graphs and machine learning techniques.<n>We introduce various neural network models, including convolutional neural networks adapted for graph data, to process these representations.<n>Our contributions include a methodology for transforming software code into code property graphs, the implementation of a convolutional neural network model for graph data, and the creation of a comprehensive dataset for training and evaluation.
  arXiv  Detail & Related papers  (2025-03-23T19:12:07Z)
• Automating the Detection of Code Vulnerabilities by Analyzing GitHub Issues [6.6681265451722895]
  We introduce a new dataset specifically designed for classifying GitHub issues relevant to vulnerability detection.<n>Results demonstrate the potential of this approach for real-world application in early vulnerability detection.<n>This work has the potential to enhance the security of open-source software ecosystems.
  arXiv  Detail & Related papers  (2025-01-09T14:13:39Z)
• Line-level Semantic Structure Learning for Code Vulnerability Detection [44.29771620061153]
  We introduce the Code Structure-Aware Network through Line-level Semantic Learning. It comprises four components: code preprocessing, global semantic awareness, line semantic awareness, and line semantic structure awareness. The CSLS model outperforms the state-of-the-art baselines in code vulnerability detection, achieving 70.57% accuracy on the Devign dataset and a 49.59% F1 score on the Reveal dataset.
  arXiv  Detail & Related papers  (2024-07-26T17:15:58Z)
• SliceLocator: Locating Vulnerable Statements with Graph-based Detectors [33.395068754566935]
  SliceLocator identifies the most relevant taint flow by selecting the highest-weighted flow path from all potential vulnerability-triggering statements.<n>We demonstrate that SliceLocator consistently performs well on four state-of-the-art GNN-based vulnerability detectors.
  arXiv  Detail & Related papers  (2024-01-05T10:15:04Z)
• CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models [58.27254444280376]
  Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure.
  arXiv  Detail & Related papers  (2023-02-08T11:54:07Z)
• VELVET: a noVel Ensemble Learning approach to automatically locate VulnErable sTatements [62.93814803258067]
  This paper presents VELVET, a novel ensemble learning approach to locate vulnerable statements in source code. Our model combines graph-based and sequence-based neural networks to successfully capture the local and global context of a program graph. VELVET achieves 99.6% and 43.6% top-1 accuracy over synthetic data and real-world data, respectively.
  arXiv  Detail & Related papers  (2021-12-20T22:45:27Z)
• Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation [57.92972327649165]
  This work explores a deep learning approach to automatically learn the insecure patterns from code corpora. Because code naturally admits graph structures with parsing, we develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program.
  arXiv  Detail & Related papers  (2021-09-07T21:24:36Z)
• Multi-context Attention Fusion Neural Network for Software Vulnerability Identification [4.05739885420409]
  We propose a deep learning model that learns to detect some of the common categories of security vulnerabilities in source code efficiently. The model builds an accurate understanding of code semantics with a lot less learnable parameters. The proposed AI achieves 98.40% F1-score on specific CWEs from the benchmarked NIST SARD dataset.
  arXiv  Detail & Related papers  (2021-04-19T11:50:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.