An Intelligent and Time-Efficient DDoS Identification Framework for Real-Time Enterprise Networks SAD-F: Spark Based Anomaly Detection Framework

• URL: http://arxiv.org/abs/2001.08155v2
• Date: Fri, 14 Feb 2020 12:19:56 GMT
• Title: An Intelligent and Time-Efficient DDoS Identification Framework for Real-Time Enterprise Networks SAD-F: Spark Based Anomaly Detection Framework
• Authors: Awais Ahmed, Sufian Hameed, Muhammad Rafi, Qublai Khan Ali Mirza
• Abstract summary: We will be exploring security analytic techniques for DDoS anomaly detection using different machine learning techniques. In this paper, we are proposing a novel approach which deals with real traffic as input to the system. We study and compare the performance factor of our proposed framework on three different testbeds.
• Score: 0.5811502603310248
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anomaly detection is a crucial step for preventing malicious activities in the network and keeping resources available all the time for legitimate users. It is noticed from various studies that classical anomaly detectors work well with small and sampled data, but the chances of failures increase with real-time (non-sampled data) traffic data. In this paper, we will be exploring security analytic techniques for DDoS anomaly detection using different machine learning techniques. In this paper, we are proposing a novel approach which deals with real traffic as input to the system. Further, we study and compare the performance factor of our proposed framework on three different testbeds including normal commodity hardware, low-end system, and high-end system. Hardware details of testbeds are discussed in the respective section. Further in this paper, we investigate the performance of the classifiers in (near) real-time detection of anomalies attacks. This study also focused on the feature selection process that is as important for the anomaly detection process as it is for general modeling problems. Several techniques have been studied for feature selection and it is observed that proper feature selection can increase performance in terms of model's execution time - which totally depends upon the traffic file or traffic capturing process.

Related papers

• Detection-Rate-Emphasized Multi-objective Evolutionary Feature Selection for Network Intrusion Detection [21.104686670216445]
  We propose DR-MOFS to model the feature selection problem in network intrusion detection as a three-objective optimization problem. In most cases, the proposed method can outperform previous methods, i.e., lead to fewer features, higher accuracy and detection rate.
  arXiv  Detail & Related papers  (2024-06-13T14:42:17Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerability [21.98595908296989]
  We demonstrate that the performance of state-of-the-art anomaly detection methods is degraded substantially by adding only small adversarial perturbations to the sensor data. We use different scoring metrics such as prediction errors, anomaly, and classification scores over several public and private datasets. We demonstrate, for the first time, the vulnerabilities of anomaly detection systems against adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-24T01:55:50Z)
• Deep Learning for Anomaly Detection in Log Data: A Survey [3.508620069426877]
  Self-learning anomaly detection techniques capture patterns in log data and report unexpected log event occurrences. Deep learning neural networks for this purpose have been presented. There exist many different architectures for deep learning and it is non-trivial to encode raw and unstructured log data.
  arXiv  Detail & Related papers  (2022-07-08T10:58:28Z)
• Sintel: A Machine Learning Framework to Extract Insights from Signals [13.04826679898367]
  We introduce Sintel, a machine learning framework for end-to-end time series tasks such as anomaly detection. Sintel logs the entire anomaly detection journey, providing detailed documentation of anomalies over time. It enables users to analyze signals, compare methods, and investigate anomalies through an interactive visualization tool.
  arXiv  Detail & Related papers  (2022-04-19T19:38:27Z)
• DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation [68.8204255655161]
  We propose a novel anomaly detection model called Discriminatory Auto-Encoder (DAE) It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase. Results show that the DAE achieves better results in both accuracy and speed of detection.
  arXiv  Detail & Related papers  (2021-09-08T14:07:55Z)
• Supervised Feature Selection Techniques in Network Intrusion Detection: a Critical Review [9.177695323629896]
  Machine Learning techniques are becoming an invaluable support for network intrusion detection. Dealing with the vast diversity and number of features that typically characterize data traffic is a hard problem. By reducing the feature space and retaining only the most significant features, Feature Selection (FS) becomes a crucial pre-processing step in network management.
  arXiv  Detail & Related papers  (2021-04-11T08:42:01Z)
• Anomaly Detection Based on Selection and Weighting in Latent Space [73.01328671569759]
  We propose a novel selection-and-weighting-based anomaly detection framework called SWAD. Experiments on both benchmark and real-world datasets have shown the effectiveness and superiority of SWAD.
  arXiv  Detail & Related papers  (2021-03-08T10:56:38Z)
• TELESTO: A Graph Neural Network Model for Anomaly Classification in Cloud Services [77.454688257702]
  Machine learning (ML) and artificial intelligence (AI) are applied on IT system operation and maintenance. One direction aims at the recognition of re-occurring anomaly types to enable remediation automation. We propose a method that is invariant to dimensionality changes of given data.
  arXiv  Detail & Related papers  (2021-02-25T14:24:49Z)
• AutoOD: Automated Outlier Detection via Curiosity-guided Search and Self-imitation Learning [72.99415402575886]
  Outlier detection is an important data mining task with numerous practical applications. We propose AutoOD, an automated outlier detection framework, which aims to search for an optimal neural network model. Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoOD achieves the best performance.
  arXiv  Detail & Related papers  (2020-06-19T18:57:51Z)
• Stance Detection Benchmark: How Robust Is Your Stance Detection? [65.91772010586605]
  Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim. We introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning setting. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets.
  arXiv  Detail & Related papers  (2020-01-06T13:37:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.