LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis

• URL: http://arxiv.org/abs/2505.18551v1
• Date: Sat, 24 May 2025 06:36:39 GMT
• Title: LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis
• Abstract summary: LAMDA is the largest and most temporally diverse Android malware benchmark to date.<n>It reflects the natural distribution and evolution of real-world Android applications.<n>It enables in-depth research into temporal drift, generalization, explainability, and evolving detection challenges.
• Score: 5.895643771545453
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning (ML)-based malware detection systems often fail to account for the dynamic nature of real-world training and test data distributions. In practice, these distributions evolve due to frequent changes in the Android ecosystem, adversarial development of new malware families, and the continuous emergence of both benign and malicious applications. Prior studies have shown that such concept drift -- distributional shifts in benign and malicious samples, leads to significant degradation in detection performance over time. Despite the practical importance of this issue, existing datasets are often outdated and limited in temporal scope, diversity of malware families, and sample scale, making them insufficient for the systematic evaluation of concept drift in malware detection. To address this gap, we present LAMDA, the largest and most temporally diverse Android malware benchmark to date, designed specifically for concept drift analysis. LAMDA spans 12 years (2013-2025, excluding 2015), includes over 1 million samples (approximately 37% labeled as malware), and covers 1,380 malware families and 150,000 singleton samples, reflecting the natural distribution and evolution of real-world Android applications. We empirically demonstrate LAMDA's utility by quantifying the performance degradation of standard ML models over time and analyzing feature stability across years. As the most comprehensive Android malware dataset to date, LAMDA enables in-depth research into temporal drift, generalization, explainability, and evolving detection challenges. The dataset and code are available at: https://iqsec-lab.github.io/LAMDA/.

Related papers

• Moirae: A Multimodal Agent Collaborative Framework for Dynamic Android Malware Detection [11.680850513898557]
  We present sysname, a collaborative framework for dynamic Android malware detection.<n>sysname collects multimodal runtime evidence and employs specialized agents to analyze complementary behavioral views.<n>Experiments on temporally and distributionally unseen datasets show that sysname achieves an accuracy of 90.06% without fine-tuning.
  arXiv  Detail & Related papers  (2026-08-28T07:03:38Z)
• McNdroid: A Longitudinal Multimodal Benchmark for Robust Drift Detection in Android Malware [24.533722488674943]
  McNdroid is the largest longitudinal multimodal Android malware benchmark for malware detection and drift analysis.<n>We evaluate standard ML and deep-learning detectors across increasing train--test time gaps.<n>We analyze modality-specific drift, malware-family evolution, and temporal changes in model explanations.
  arXiv  Detail & Related papers  (2026-05-07T19:53:24Z)
• Empirical Evaluation of SMOTE in Android Malware Detection with Machine Learning: Challenges and Performance in CICMalDroid 2020 [0.0]
  This work tests Machine Learning algorithms in detecting malicious code from dynamic execution characteristics.<n>In 75% of the tested configurations, the application of SMOTE led to performance degradation or only marginal improvements.<n>Tree-based algorithms, such as XGBoost and Random Forest, consistently outperformed the others, achieving weighted recall above 94%.
  arXiv  Detail & Related papers  (2026-02-09T14:47:47Z)
• Out-of-Distribution Detection for Continual Learning: Design Principles and Benchmarking [44.75780122845172]
  Recent years have witnessed significant progress in the development of machine learning models across a wide range of fields.<n>As these models are deployed in ever-changing real-world scenarios, their ability to remain reliable and adaptive over time becomes increasingly important.
  arXiv  Detail & Related papers  (2025-12-16T22:50:01Z)
• CITADEL: A Semi-Supervised Active Learning Framework for Malware Detection Under Continuous Distribution Drift [2.6040598178524106]
  We propose CITADEL, a robust semi-supervised active learning framework for Android malware detection.<n>We introduce malware-specific augmentations, Bernoulli bit flips and masking, that simulate realistic drift behaviors.<n>We show that CITADEL outperforms prior work, achieving F1 score of over 1%, 3%, 7%, and 14% respectively, using only 40% labeled samples.
  arXiv  Detail & Related papers  (2025-11-15T01:34:15Z)
• ADAPT: A Pseudo-labeling Approach to Combat Concept Drift in Malware Detection [0.8192907805418583]
  Adapting machine learning models to changing data distributions requires frequent updates.<n>We introduce texttADAPT, a novel pseudo-labeling semi-supervised algorithm for addressing concept drift.
  arXiv  Detail & Related papers  (2025-07-11T13:47:07Z)
• MalVis: A Large-Scale Image-Based Framework and Dataset for Advancing Android Malware Classification [2.156165260537145]
  MalVis is a unified visualization framework that integrates entropy and N-gram analysis to emphasize structural and anomalous patterns in malware bytecode.<n>We evaluate MalVis against state-of-the-art visualization techniques using leading CNN models.<n>MalVis achieves strong results: 95.19% accuracy, 90.81% F1-score, 92.58% precision, 89.10% recall, 87.58% MCC, and 98.06% ROC-AUC.
  arXiv  Detail & Related papers  (2025-05-17T18:19:35Z)
• Are Anomaly Scores Telling the Whole Story? A Benchmark for Multilevel Anomaly Detection [46.244213695024]
  Anomaly detection (AD) is a machine learning task that identifies anomalies by learning patterns from normal training data. Existing models primarily operate in a binary setting, and the anomaly scores they produce are usually based on the deviation of data points from normal data. We propose a novel setting, Multilevel AD (MAD), in which the anomaly score represents the severity of anomalies in real-world applications. Second, we introduce a novel benchmark, MAD-Bench, that evaluates models not only on their ability to detect anomalies, but also on how effectively their anomaly scores reflect severity.
  arXiv  Detail & Related papers  (2024-11-21T14:18:37Z)
• Revisiting Static Feature-Based Android Malware Detection [0.8192907805418583]
  This paper highlights critical pitfalls that undermine the validity of machine learning research in Android malware detection. We propose solutions for improving datasets and methodological practices, enabling fairer model comparisons. Our paper aims to support future research in Android malware detection and other security domains, enhancing the reliability and validity of published results.
  arXiv  Detail & Related papers  (2024-09-11T16:37:50Z)
• Revisiting Concept Drift in Windows Malware Detection: Adaptation to Real Drifted Malware with Minimal Samples [10.352741619176383]
  We propose a new technique for detecting and classifying drifted malware.<n>It learns drift-invariant features in malware control flow graphs by leveraging graph neural networks with adversarial domain adaptation.<n>Our approach significantly improves drifted malware detection on publicly available benchmarks and real-world malware databases reported daily by security companies.
  arXiv  Detail & Related papers  (2024-07-18T22:06:20Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [52.228708947607636]
  This paper introduces a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods. The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics. We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection [44.21198064126152]
  We propose a novel anomaly detection framework named ImDiffusion. ImDiffusion combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets.
  arXiv  Detail & Related papers  (2023-07-03T04:57:40Z)
• DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness [58.23214712926585]
  We develop a certified defense, DRSM (De-Randomized Smoothed MalConv), by redesigning the de-randomized smoothing technique for the domain of malware detection. Specifically, we propose a window ablation scheme to provably limit the impact of adversarial bytes while maximally preserving local structures of the executables. We are the first to offer certified robustness in the realm of static detection of malware executables.
  arXiv  Detail & Related papers  (2023-03-20T17:25:22Z)
• Investigating Feature and Model Importance in Android Malware Detection: An Implemented Survey and Experimental Comparison of ML-Based Methods [2.9248916859490173]
  We show that high detection accuracies can be achieved using features extracted through static analysis alone. Random forests are generally the most effective model, outperforming more complex deep learning approaches.
  arXiv  Detail & Related papers  (2023-01-30T10:48:10Z)
• Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time [69.77704012415845]
  Temporal shifts can considerably degrade performance of machine learning models deployed in the real world. We benchmark 13 prior approaches, including methods in domain generalization, continual learning, self-supervised learning, and ensemble learning. Under both evaluation strategies, we observe an average performance drop of 20% from in-distribution to out-of-distribution data.
  arXiv  Detail & Related papers  (2022-11-25T17:07:53Z)
• Towards a Fair Comparison and Realistic Design and Evaluation Framework of Android Malware Detectors [63.75363908696257]
  We analyze 10 influential research works on Android malware detection using a common evaluation framework. We identify five factors that, if not taken into account when creating datasets and designing detectors, significantly affect the trained ML models. We conclude that the studied ML-based detectors have been evaluated optimistically, which justifies the good published results.
  arXiv  Detail & Related papers  (2022-05-25T08:28:08Z)
• Fast & Furious: Modelling Malware Detection as Evolving Data Streams [6.6892028759947175]
  Malware is a major threat to computer systems and imposes many challenges to cyber security. In this work, we evaluate the impact of concept drift on malware classifiers for two Android datasets.
  arXiv  Detail & Related papers  (2022-05-24T18:43:40Z)
• TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks [73.01104041298031]
  TadGAN is an unsupervised anomaly detection approach built on Generative Adversarial Networks (GANs) To capture the temporal correlations of time series, we use LSTM Recurrent Neural Networks as base models for Generators and Critics. To demonstrate the performance and generalizability of our approach, we test several anomaly scoring techniques and report the best-suited one.
  arXiv  Detail & Related papers  (2020-09-16T15:52:04Z)
• Stance Detection Benchmark: How Robust Is Your Stance Detection? [65.91772010586605]
  Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim. We introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning setting. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets.
  arXiv  Detail & Related papers  (2020-01-06T13:37:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.