EMBERSim: A Large-Scale Databank for Boosting Similarity Search in Malware Analysis

• URL: http://arxiv.org/abs/2310.01835v1
• Date: Tue, 3 Oct 2023 06:58:45 GMT
• Title: EMBERSim: A Large-Scale Databank for Boosting Similarity Search in Malware Analysis
• Authors: Dragos Georgian Corlatescu, Alexandru Dinu, Mihaela Gaman, Paul Sumedrea
• Abstract summary: In recent years there has been a shift from quantifications-based malware detection towards machine learning. We propose to address the deficiencies in the space of similarity research on binary files, starting from EMBER. We enhance EMBER with similarity information as well as malware class tags, to enable further research in the similarity space.
• Score: 48.5877840394508
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years there has been a shift from heuristics-based malware detection towards machine learning, which proves to be more robust in the current heavily adversarial threat landscape. While we acknowledge machine learning to be better equipped to mine for patterns in the increasingly high amounts of similar-looking files, we also note a remarkable scarcity of the data available for similarity-targeted research. Moreover, we observe that the focus in the few related works falls on quantifying similarity in malware, often overlooking the clean data. This one-sided quantification is especially dangerous in the context of detection bypass. We propose to address the deficiencies in the space of similarity research on binary files, starting from EMBER - one of the largest malware classification data sets. We enhance EMBER with similarity information as well as malware class tags, to enable further research in the similarity space. Our contribution is threefold: (1) we publish EMBERSim, an augmented version of EMBER, that includes similarity-informed tags; (2) we enrich EMBERSim with automatically determined malware class tags using the open-source tool AVClass on VirusTotal data and (3) we describe and share the implementation for our class scoring technique and leaf similarity method.

Related papers

• Multi-label Classification for Android Malware Based on Active Learning [7.599125552187342]
  We propose MLCDroid, an ML-based multi-label classification approach that can directly indicate the existence of pre-defined malicious behaviors. We compare the results of 70 algorithm combinations to evaluate the effectiveness (best at 73.3%). This is the first multi-label Android malware classification approach intending to provide more information on fine-grained malicious behaviors.
  arXiv  Detail & Related papers  (2024-10-09T01:09:24Z)
• A Survey of Malware Detection Using Deep Learning [6.349503549199403]
  This paper investigates advances in malware detection on Windows, iOS, Android, and Linux using deep learning (DL) We discuss the issues and the challenges in malware detection using DL classifiers. We examine eight popular DL approaches on various datasets.
  arXiv  Detail & Related papers  (2024-07-27T02:49:55Z)
• Small Effect Sizes in Malware Detection? Make Harder Train/Test Splits! [51.668411293817464]
  Industry practitioners care about small improvements in malware detection accuracy because their models are deployed to hundreds of millions of machines. Academic research is often restrained to public datasets on the order of ten thousand samples. We devise an approach to generate a benchmark of difficulty from a pool of available samples.
  arXiv  Detail & Related papers  (2023-12-25T21:25:55Z)
• Towards a Fair Comparison and Realistic Design and Evaluation Framework of Android Malware Detectors [63.75363908696257]
  We analyze 10 influential research works on Android malware detection using a common evaluation framework. We identify five factors that, if not taken into account when creating datasets and designing detectors, significantly affect the trained ML models. We conclude that the studied ML-based detectors have been evaluated optimistically, which justifies the good published results.
  arXiv  Detail & Related papers  (2022-05-25T08:28:08Z)
• Using Static and Dynamic Malware features to perform Malware Ascription [0.0]
  We employ various Static and Dynamic features of malicious executables to classify malware based on their family. We leverage Cuckoo Sandbox and machine learning to make progress in this research.
  arXiv  Detail & Related papers  (2021-12-05T18:01:09Z)
• New Datasets for Dynamic Malware Classification [0.0]
  We introduce two new, updated datasets of malicious software, VirusSamples and VirusShare. This paper analyzes multi-class malware classification performance of the balanced and imbalanced version of these two datasets. Results show that Support Vector Machine, achieves the highest score of 94% in the imbalanced VirusSample dataset. XGBoost, one of the most common gradient boosting-based models, achieves the highest score of 90% and 80%.in both versions of the VirusShare dataset.
  arXiv  Detail & Related papers  (2021-11-30T08:31:16Z)
• MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labels [21.050311121388813]
  We have created the Malware Open-source Threat Intelligence Family (MOTIF) dataset. MOTIF contains 3,095 malware samples from 454 families, making it the largest and most diverse public malware dataset. We provide aliases of the different names used to describe the same malware family, allowing us to benchmark for the first time accuracy of existing tools.
  arXiv  Detail & Related papers  (2021-11-29T23:59:50Z)
• S3M: Siamese Stack (Trace) Similarity Measure [55.58269472099399]
  We present S3M -- the first approach to computing stack trace similarity based on deep learning. It is based on a biLSTM encoder and a fully-connected classifier to compute similarity. Our experiments demonstrate the superiority of our approach over the state-of-the-art on both open-sourced data and a private JetBrains dataset.
  arXiv  Detail & Related papers  (2021-03-18T21:10:41Z)
• DecAug: Augmenting HOI Detection via Decomposition [54.65572599920679]
  Current algorithms suffer from insufficient training samples and category imbalance within datasets. We propose an efficient and effective data augmentation method called DecAug for HOI detection. Experiments show that our method brings up to 3.3 mAP and 1.6 mAP improvements on V-COCO and HICODET dataset.
  arXiv  Detail & Related papers  (2020-10-02T13:59:05Z)
• Adversarial EXEmples: A Survey and Experimental Evaluation of Practical Attacks on Machine Learning for Windows Malware Detection [67.53296659361598]
  adversarial EXEmples can bypass machine learning-based detection by perturbing relatively few input bytes. We develop a unifying framework that does not only encompass and generalize previous attacks against machine-learning models, but also includes three novel attacks. These attacks, named Full DOS, Extend and Shift, inject the adversarial payload by respectively manipulating the DOS header, extending it, and shifting the content of the first section.
  arXiv  Detail & Related papers  (2020-08-17T07:16:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.