False Sense of Security: Leveraging XAI to Analyze the Reasoning and True Performance of Context-less DGA Classifiers

• URL: http://arxiv.org/abs/2307.04358v2
• Date: Mon, 25 Sep 2023 06:07:56 GMT
• Title: False Sense of Security: Leveraging XAI to Analyze the Reasoning and True Performance of Context-less DGA Classifiers
• Authors: Arthur Drichel and Ulrike Meyer
• Abstract summary: Domain Generation Algorithm (DGA) detection seems to be solved, considering that available deep learning classifiers achieve accuracies of over 99.9%. These classifiers provide a false sense of security as they are heavily biased and allow for trivial detection bypass. In this work, we leverage explainable artificial intelligence (XAI) methods to analyze the reasoning of deep learning classifiers.
• Score: 1.930852251165745
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The problem of revealing botnet activity through Domain Generation Algorithm (DGA) detection seems to be solved, considering that available deep learning classifiers achieve accuracies of over 99.9%. However, these classifiers provide a false sense of security as they are heavily biased and allow for trivial detection bypass. In this work, we leverage explainable artificial intelligence (XAI) methods to analyze the reasoning of deep learning classifiers and to systematically reveal such biases. We show that eliminating these biases from DGA classifiers considerably deteriorates their performance. Nevertheless we are able to design a context-aware detection system that is free of the identified biases and maintains the detection rate of state-of-the art deep learning classifiers. In this context, we propose a visual analysis system that helps to better understand a classifier's reasoning, thereby increasing trust in and transparency of detection methods and facilitating decision-making.

Related papers

• Classifier Guidance Enhances Diffusion-based Adversarial Purification by Preserving Predictive Information [75.36597470578724]
  Adversarial purification is one of the promising approaches to defend neural networks against adversarial attacks. We propose gUided Purification (COUP) algorithm, which purifies while keeping away from the classifier decision boundary. Experimental results show that COUP can achieve better adversarial robustness under strong attack methods.
  arXiv  Detail & Related papers  (2024-08-12T02:48:00Z)
• Towards Robust Domain Generation Algorithm Classification [1.4542411354617986]
  We implement 32 white-box attacks, 19 of which are very effective and induce a false-negative rate (FNR) of $approx$ 100% on unhardened classifiers. We propose a novel training scheme that leverages adversarial latent space vectors and discretized adversarial domains to significantly improve robustness.
  arXiv  Detail & Related papers  (2024-04-09T11:56:29Z)
• XAI-Based Detection of Adversarial Attacks on Deepfake Detectors [0.0]
  We introduce a novel methodology for identifying adversarial attacks on deepfake detectors using XAI. Our approach contributes not only to the detection of deepfakes but also enhances the understanding of possible adversarial attacks.
  arXiv  Detail & Related papers  (2024-03-05T13:25:30Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Performance evaluation of Machine learning algorithms for Intrusion Detection System [0.40964539027092917]
  This paper focuses on intrusion detection systems (IDSs) analysis using Machine Learning (ML) techniques. We analyze the KDD CUP-'99' intrusion detection dataset used for training and validating ML models.
  arXiv  Detail & Related papers  (2023-10-01T06:35:37Z)
• Anomaly Detection in Cybersecurity: Unsupervised, Graph-Based and Supervised Learning Methods in Adversarial Environments [63.942632088208505]
  Inherent to today's operating environment is the practice of adversarial machine learning. In this work, we examine the feasibility of unsupervised learning and graph-based methods for anomaly detection. We incorporate a realistic adversarial training mechanism when training our supervised models to enable strong classification performance in adversarial environments.
  arXiv  Detail & Related papers  (2021-05-14T10:05:10Z)
• Detection of Adversarial Supports in Few-shot Classifiers Using Feature Preserving Autoencoders and Self-Similarity [89.26308254637702]
  We propose a detection strategy to highlight adversarial support sets. We make use of feature preserving autoencoder filtering and also the concept of self-similarity of a support set to perform this detection. Our method is attack-agnostic and also the first to explore detection for few-shot classifiers to the best of our knowledge.
  arXiv  Detail & Related papers  (2020-12-09T14:13:41Z)
• FairCVtest Demo: Understanding Bias in Multimodal Learning with a Testbed in Fair Automatic Recruitment [79.23531577235887]
  This demo shows the capacity of the Artificial Intelligence (AI) behind a recruitment tool to extract sensitive information from unstructured data. Aditionally, the demo includes a new algorithm for discrimination-aware learning which eliminates sensitive information in our multimodal AI framework.
  arXiv  Detail & Related papers  (2020-09-12T17:45:09Z)
• Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection [66.05992706105224]
  In this paper, an effective anomaly detection framework is proposed utilizing Bayesian Optimization technique. The performance of the considered algorithms is evaluated using the ISCX 2012 dataset. Experimental results show the effectiveness of the proposed framework in term of accuracy rate, precision, low-false alarm rate, and recall.
  arXiv  Detail & Related papers  (2020-08-05T19:29:35Z)
• Analyzing the Real-World Applicability of DGA Classifiers [3.0969191504482243]
  We propose a novel classifier for separating benign domains from domains generated by DGAs. We evaluate their classification performance and compare them with respect to explainability, robustness, and training and classification speed. Our newly proposed binary classifier generalizes well to other networks, is time-robust, and able to identify previously unknown DGAs.
  arXiv  Detail & Related papers  (2020-06-19T12:34:05Z)
• Do Gradient-based Explanations Tell Anything About Adversarial Robustness to Android Malware? [20.11888851905904]
  We investigate whether gradient-based attribution methods can be used to help identify and select more robust algorithms. Experiments conducted on two different datasets and five classification algorithms for Android malware detection show that a strong connection exists between the uniformity of explanations and adversarial robustness.
  arXiv  Detail & Related papers  (2020-05-04T13:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.