An Ensemble of Pre-trained Transformer Models For Imbalanced Multiclass Malware Classification

• URL: http://arxiv.org/abs/2112.13236v1
• Date: Sat, 25 Dec 2021 13:40:07 GMT
• Title: An Ensemble of Pre-trained Transformer Models For Imbalanced Multiclass Malware Classification
• Authors: Ferhat Demirk{\i}ran, Aykut \c{C}ay{\i}r, U\u{g}ur \"Unal, Hasan Da\u{g}
• Abstract summary: API call sequences made by malware are widely utilized features by machine and deep learning models for malware classification. Traditional machine and deep learning models remain incapable of capturing sequence relationships between API calls. Our experiments demonstrate that the transformer model with one transformer block layer surpassed the widely used base architecture, LSTM. BERT or CANINE, pre-trained transformer models, outperformed in classifying highly imbalanced malware families according to evaluation metrics, F1-score, and AUC score.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classification of malware families is crucial for a comprehensive understanding of how they can infect devices, computers, or systems. Thus, malware identification enables security researchers and incident responders to take precautions against malware and accelerate mitigation. API call sequences made by malware are widely utilized features by machine and deep learning models for malware classification as these sequences represent the behavior of malware. However, traditional machine and deep learning models remain incapable of capturing sequence relationships between API calls. On the other hand, the transformer-based models process sequences as a whole and learn relationships between API calls due to multi-head attention mechanisms and positional embeddings. Our experiments demonstrate that the transformer model with one transformer block layer surpassed the widely used base architecture, LSTM. Moreover, BERT or CANINE, pre-trained transformer models, outperformed in classifying highly imbalanced malware families according to evaluation metrics, F1-score, and AUC score. Furthermore, the proposed bagging-based random transformer forest (RTF), an ensemble of BERT or CANINE, has reached the state-of-the-art evaluation scores on three out of four datasets, particularly state-of-the-art F1-score of 0.6149 on one of the commonly used benchmark dataset.

Related papers

• Toward Multi-class Anomaly Detection: Exploring Class-aware Unified Model against Inter-class Interference [67.36605226797887]
  We introduce a Multi-class Implicit Neural representation Transformer for unified Anomaly Detection (MINT-AD) By learning the multi-class distributions, the model generates class-aware query embeddings for the transformer decoder. MINT-AD can project category and position information into a feature embedding space, further supervised by classification and prior probability loss functions.
  arXiv  Detail & Related papers  (2024-03-21T08:08:31Z)
• Semi-supervised Classification of Malware Families Under Extreme Class Imbalance via Hierarchical Non-Negative Matrix Factorization with Automatic Model Selection [34.7994627734601]
  We propose a novel hierarchical semi-supervised algorithm, which can be used in the early stages of the malware family labeling process. With HNMFk, we exploit the hierarchical structure of the malware data together with a semi-supervised setup, which enables us to classify malware families under conditions of extreme class imbalance. Our solution can perform abstaining predictions, or rejection option, which yields promising results in the identification of novel malware families.
  arXiv  Detail & Related papers  (2023-09-12T23:45:59Z)
• STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognition [50.064502884594376]
  We study the problem of human action recognition using motion capture (MoCap) sequences. We propose a novel Spatial-Temporal Mesh Transformer (STMT) to directly model the mesh sequences. The proposed method achieves state-of-the-art performance compared to skeleton-based and point-cloud-based models.
  arXiv  Detail & Related papers  (2023-03-31T16:19:27Z)
• Reliable Malware Analysis and Detection using Topology Data Analysis [12.031113181911627]
  Malwares are becoming more complex and they are spreading on networks targeting different infrastructures and personal-end devices. To defend against malwares, recent work has proposed different techniques based on signatures and machine learning.
  arXiv  Detail & Related papers  (2022-11-03T00:46:52Z)
• Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection [122.4894940892536]
  We present a novel self-supervised masked convolutional transformer block (SSMCTB) that comprises the reconstruction-based functionality at a core architectural level. In this work, we extend our previous self-supervised predictive convolutional attentive block (SSPCAB) with a 3D masked convolutional layer, a transformer for channel-wise attention, as well as a novel self-supervised objective based on Huber loss.
  arXiv  Detail & Related papers  (2022-09-25T04:56:10Z)
• Self-Supervised Vision Transformers for Malware Detection [0.0]
  This paper presents SHERLOCK, a self-supervision based deep learning model to detect malware based on the Vision Transformer (ViT) architecture. Our proposed model is also able to outperform state-of-the-art techniques for multi-class malware classification of types and family with macro-F1 score of.497 and.491 respectively.
  arXiv  Detail & Related papers  (2022-08-15T07:49:58Z)
• Benchmarking Machine Learning Robustness in Covid-19 Genome Sequence Classification [109.81283748940696]
  We introduce several ways to perturb SARS-CoV-2 genome sequences to mimic the error profiles of common sequencing platforms such as Illumina and PacBio. We show that some simulation-based approaches are more robust (and accurate) than others for specific embedding methods to certain adversarial attacks to the input sequences.
  arXiv  Detail & Related papers  (2022-07-18T19:16:56Z)
• Fast & Furious: Modelling Malware Detection as Evolving Data Streams [6.6892028759947175]
  Malware is a major threat to computer systems and imposes many challenges to cyber security. In this work, we evaluate the impact of concept drift on malware classifiers for two Android datasets.
  arXiv  Detail & Related papers  (2022-05-24T18:43:40Z)
• TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery [60.439434751619736]
  We propose TraSeTR, a Track-to-Segment Transformer that exploits tracking cues to assist surgical instrument segmentation. TraSeTR jointly reasons about the instrument type, location, and identity with instance-level predictions. The effectiveness of our method is demonstrated with state-of-the-art instrument type segmentation results on three public datasets.
  arXiv  Detail & Related papers  (2022-02-17T05:52:18Z)
• MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection [0.0]
  Transformers, a category of attention-based deep learning techniques, have recently shown impressive results in solving different tasks. We propose a model based on BERT (Bi Representations from Transformers) which performs a static analysis on the source code of Android applications. The obtained results are promising and show the high performance obtained by Transformer-based models for malicious software detection.
  arXiv  Detail & Related papers  (2021-03-05T17:09:46Z)
• Transfer Learning without Knowing: Reprogramming Black-box Machine Learning Models with Scarce Data and Limited Resources [78.72922528736011]
  We propose a novel approach, black-box adversarial reprogramming (BAR), that repurposes a well-trained black-box machine learning model. Using zeroth order optimization and multi-label mapping techniques, BAR can reprogram a black-box ML model solely based on its input-output responses. BAR outperforms state-of-the-art methods and yields comparable performance to the vanilla adversarial reprogramming method.
  arXiv  Detail & Related papers  (2020-07-17T01:52:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.