Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code

• URL: http://arxiv.org/abs/2409.17513v1
• Date: Thu, 26 Sep 2024 03:48:47 GMT
• Title: Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code
• Authors: Gary A. McCully, John D. Hastings, Shengjie Xu, Adam Fortier
• Abstract summary: This research investigates the application of unidirectional transformer-based embeddings, specifically GPT-2. Our study reveals that embeddings from GPT-2 model significantly outperform those from bidirectional models of BERT and RoBERTa.
• Score: 5.4141465747474475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ransomware and other forms of malware cause significant financial and operational damage to organizations by exploiting long-standing and often difficult-to-detect software vulnerabilities. To detect vulnerabilities such as buffer overflows in compiled code, this research investigates the application of unidirectional transformer-based embeddings, specifically GPT-2. Using a dataset of LLVM functions, we trained a GPT-2 model to generate embeddings, which were subsequently used to build LSTM neural networks to differentiate between vulnerable and non-vulnerable code. Our study reveals that embeddings from the GPT-2 model significantly outperform those from bidirectional models of BERT and RoBERTa, achieving an accuracy of 92.5% and an F1-score of 89.7%. LSTM neural networks were developed with both frozen and unfrozen embedding model layers. The model with the highest performance was achieved when the embedding layers were unfrozen. Further, the research finds that, in exploring the impact of different optimizers within this domain, the SGD optimizer demonstrates superior performance over Adam. Overall, these findings reveal important insights into the potential of unidirectional transformer-based approaches in enhancing cybersecurity defenses.

Related papers

• Hybrid Deep Convolutional Neural Networks Combined with Autoencoders And Augmented Data To Predict The Look-Up Table 2006 [2.082445711353476]
  This study explores the development of a hybrid deep convolutional neural network (DCNN) model enhanced by autoencoders and data augmentation techniques. By augmenting the original input features using three different autoencoder configurations, the model's predictive capabilities were significantly improved.
  arXiv  Detail & Related papers  (2024-08-26T20:45:07Z)
• Enhancing IoT Security with CNN and LSTM-Based Intrusion Detection Systems [0.23408308015481666]
  Our proposed model consists on a combination of convolutional neural network (CNN) and long short-term memory (LSTM) deep learning (DL) models. This fusion facilitates the detection and classification of IoT traffic into binary categories, benign and malicious activities. Our proposed model achieves an accuracy rate of 98.42%, accompanied by a minimal loss of 0.0275.
  arXiv  Detail & Related papers  (2024-05-28T22:12:15Z)
• SecureBERT and LLAMA 2 Empowered Control Area Network Intrusion Detection and Classification [2.824211356106516]
  We develop two distinct models for CAN intrusion detection: CAN-SecureBERT and CAN-LLAMA2. Can-LLAMA2 model surpasses the state-of-the-art models by achieving an exceptional performance 0.999993 in terms of balanced accuracy, precision detection rate, F1 score, and a remarkably low false alarm rate of 3.10e-6.
  arXiv  Detail & Related papers  (2023-11-19T23:49:08Z)
• LogShield: A Transformer-based APT Detection System Leveraging Self-Attention [2.1256044139613772]
  This paper proposes LogShield, a framework designed to detect APT attack patterns leveraging the power of self-attention in transformers. We incorporate customized embedding layers to effectively capture the context of event sequences derived from provenance graphs. Our framework achieved superior F1 scores of 98% and 95% on the two datasets respectively, surpassing the F1 scores of 96% and 94% obtained by LSTM models.
  arXiv  Detail & Related papers  (2023-11-09T20:43:15Z)
• MTS-DVGAN: Anomaly Detection in Cyber-Physical Systems using a Dual Variational Generative Adversarial Network [7.889342625283858]
  Deep generative models are promising in detecting novel cyber-physical attacks, mitigating the vulnerability of Cyber-physical systems (CPSs) without relying on labeled information. This article proposes a novel unsupervised dual variational generative adversarial model named MST-DVGAN. The central concept is to enhance the model's discriminative capability by widening the distinction between reconstructed abnormal samples and their normal counterparts.
  arXiv  Detail & Related papers  (2023-11-04T11:19:03Z)
• Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting [95.82600221180415]
  Human forecasting in crowds presents the challenges of modelling social interactions and outputting collision-free multimodal distribution. We introduce SGANv2, an improved safety-compliant SGAN architecture equipped with motion-temporal interaction modelling and a transformer-based discriminator design.
  arXiv  Detail & Related papers  (2022-09-25T15:18:56Z)
• Evaluating the Adversarial Robustness for Fourier Neural Operators [78.36413169647408]
  Fourier Neural Operator (FNO) was the first to simulate turbulent flow with zero-shot super-resolution. We generate adversarial examples for FNO based on norm-bounded data input perturbations. Our results show that the model's robustness degrades rapidly with increasing perturbation levels.
  arXiv  Detail & Related papers  (2022-04-08T19:19:42Z)
• Anomaly Detection in Cybersecurity: Unsupervised, Graph-Based and Supervised Learning Methods in Adversarial Environments [63.942632088208505]
  Inherent to today's operating environment is the practice of adversarial machine learning. In this work, we examine the feasibility of unsupervised learning and graph-based methods for anomaly detection. We incorporate a realistic adversarial training mechanism when training our supervised models to enable strong classification performance in adversarial environments.
  arXiv  Detail & Related papers  (2021-05-14T10:05:10Z)
• Uncertainty-Aware Deep Calibrated Salient Object Detection [74.58153220370527]
  Existing deep neural network based salient object detection (SOD) methods mainly focus on pursuing high network accuracy. These methods overlook the gap between network accuracy and prediction confidence, known as the confidence uncalibration problem. We introduce an uncertaintyaware deep SOD network, and propose two strategies to prevent deep SOD networks from being overconfident.
  arXiv  Detail & Related papers  (2020-12-10T23:28:36Z)
• MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation [153.56211546576978]
  In this work, we propose that better soft targets with higher compatibil-ity can be generated by using a label generator. We can employ the meta-learning technique to optimize this label generator. The experiments are conducted on two standard classificationbenchmarks, namely CIFAR-100 and ILSVRC2012.
  arXiv  Detail & Related papers  (2020-08-27T13:04:27Z)
• Graph Backdoor [53.70971502299977]
  We present GTA, the first backdoor attack on graph neural networks (GNNs) GTA departs in significant ways: it defines triggers as specific subgraphs, including both topological structures and descriptive features. It can be instantiated for both transductive (e.g., node classification) and inductive (e.g., graph classification) tasks.
  arXiv  Detail & Related papers  (2020-06-21T19:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.