SoK: Leveraging Transformers for Malware Analysis

• URL: http://arxiv.org/abs/2405.17190v1
• Date: Mon, 27 May 2024 14:14:07 GMT
• Title: SoK: Leveraging Transformers for Malware Analysis
• Authors: Pradip Kunwar, Kshitiz Aryal, Maanak Gupta, Mahmoud Abdelsalam, Elisa Bertino,
• Abstract summary: The introduction of transformers has been an important breakthrough for AI research and application as transformers are the foundation behind Generative AI. A promising application domain for transformers is cybersecurity, in particular the malware domain analysis. This SoK paper aims to provide a comprehensive analysis of transformer-based approaches designed for malware analysis.
• Score: 8.999677363643224
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The introduction of transformers has been an important breakthrough for AI research and application as transformers are the foundation behind Generative AI. A promising application domain for transformers is cybersecurity, in particular the malware domain analysis. The reason is the flexibility of the transformer models in handling long sequential features and understanding contextual relationships. However, as the use of transformers for malware analysis is still in the infancy stage, it is critical to evaluate, systematize, and contextualize existing literature to foster future research. This Systematization of Knowledge (SoK) paper aims to provide a comprehensive analysis of transformer-based approaches designed for malware analysis. Based on our systematic analysis of existing knowledge, we structure and propose taxonomies based on: (a) how different transformers are adapted, organized, and modified across various use cases; and (b) how diverse feature types and their representation capabilities are reflected. We also provide an inventory of datasets used to explore multiple research avenues in the use of transformers for malware analysis and discuss open challenges with future research directions. We believe that this SoK paper will assist the research community in gaining detailed insights from existing work and will serve as a foundational resource for implementing novel research using transformers for malware analysis.

Related papers

• Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey [0.3108011671896571]
  This survey paper provides a comprehensive analysis of the utilization of Transformers and LLMs in cyber-threat detection systems. The fundamentals of Transformers are discussed, including background information on various cyber-attacks and datasets commonly used in this field. It explores the diverse environments and applications where Transformers and LLMs-based IDS have been implemented, including computer networks, IoT devices, critical infrastructure protection, cloud computing, SDN, as well as in autonomous vehicles.
  arXiv  Detail & Related papers  (2024-08-14T14:28:11Z)
• Survey: Transformer-based Models in Data Modality Conversion [0.8136541584281987]
  Modality Conversion involves the transformation of data from one form of representation to another, mimicking the way humans integrate and interpret sensory information. This paper provides a comprehensive review of transformer-based models applied to the primary modalities of text, vision, and speech, discussing their architectures, conversion methodologies, and applications.
  arXiv  Detail & Related papers  (2024-08-08T18:39:14Z)
• Transformers in Reinforcement Learning: A Survey [7.622978576824539]
  Transformers have impacted domains like natural language processing, computer vision, and robotics, where they improve performance compared to other neural networks. This survey explores how transformers are used in reinforcement learning (RL), where they are seen as a promising solution for addressing challenges such as unstable training, credit assignment, lack of interpretability, and partial observability.
  arXiv  Detail & Related papers  (2023-07-12T07:51:12Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review [6.953789750981636]
  We provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks.
  arXiv  Detail & Related papers  (2023-01-09T16:56:23Z)
• A Survey on Transformers in Reinforcement Learning [66.23773284875843]
  Transformer has been considered the dominating neural architecture in NLP and CV, mostly under supervised settings. Recently, a similar surge of using Transformers has appeared in the domain of reinforcement learning (RL), but it is faced with unique design choices and challenges brought by the nature of RL. This paper systematically reviews motivations and progress on using Transformers in RL, provide a taxonomy on existing works, discuss each sub-field, and summarize future prospects.
  arXiv  Detail & Related papers  (2023-01-08T14:04:26Z)
• Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection [119.93025368028083]
  We design a novel Transformer-style Human-Object Interaction (HOI) detector, i.e., Structure-aware Transformer over Interaction Proposals (STIP) STIP decomposes the process of HOI set prediction into two subsequent phases, i.e., an interaction proposal generation is first performed, and then followed by transforming the non-parametric interaction proposals into HOI predictions via a structure-aware Transformer. The structure-aware Transformer upgrades vanilla Transformer by encoding additionally the holistically semantic structure among interaction proposals as well as the locally spatial structure of human/object within each interaction proposal, so as to strengthen HOI
  arXiv  Detail & Related papers  (2022-06-13T16:21:08Z)
• Transformers in Time Series: A Survey [66.50847574634726]
  We systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification.
  arXiv  Detail & Related papers  (2022-02-15T01:43:27Z)
• Transformers in Vision: A Survey [101.07348618962111]
  Transformers enable modeling long dependencies between input sequence elements and support parallel processing of sequence. Transformers require minimal inductive biases for their design and are naturally suited as set-functions. This survey aims to provide a comprehensive overview of the Transformer models in the computer vision discipline.
  arXiv  Detail & Related papers  (2021-01-04T18:57:24Z)
• A Survey on Visual Transformer [126.56860258176324]
  Transformer is a type of deep neural network mainly based on the self-attention mechanism. In this paper, we review these vision transformer models by categorizing them in different tasks and analyzing their advantages and disadvantages.
  arXiv  Detail & Related papers  (2020-12-23T09:37:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.