Filtering DDoS Attacks from Unlabeled Network Traffic Data Using Online Deep Learning

• URL: http://arxiv.org/abs/2012.06805v1
• Date: Sat, 12 Dec 2020 12:39:05 GMT
• Title: Filtering DDoS Attacks from Unlabeled Network Traffic Data Using Online Deep Learning
• Authors: Wesley Joon-Wie Tann, Jackie Tan Jin Wei, Joanna Purba, Ee-Chien Chang
• Abstract summary: DDoS attacks are simple, effective, and still pose a significant threat even after more than two decades. There are challenges in adopting deep learning solutions due to the ever-changing profiles, the lack of labeled data, and constraints in the online setting. In this paper, we propose two frameworks that utilize both the historic $mathcal N$ and the mixture $mathcal M$ traffic obtained during attacks.
• Score: 7.683035842994701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: DDoS attacks are simple, effective, and still pose a significant threat even after more than two decades. Given the recent success in machine learning, it is interesting to investigate how we can leverage deep learning to filter out application layer attack requests. There are challenges in adopting deep learning solutions due to the ever-changing profiles, the lack of labeled data, and constraints in the online setting. Offline unsupervised learning methods can sidestep these hurdles by learning an anomaly detector $N$ from the normal-day traffic ${\mathcal N}$. However, anomaly detection does not exploit information acquired during attacks, and their performance typically is not satisfactory. In this paper, we propose two frameworks that utilize both the historic ${\mathcal N}$ and the mixture ${\mathcal M}$ traffic obtained during attacks, consisting of unlabeled requests. We also introduce a machine learning optimization problem that aims to sift out the attacks using ${\mathcal N}$ and ${\mathcal M}$. First, our proposed approach, inspired by statistical methods, extends an unsupervised anomaly detector $N$ to solve the problem using estimated conditional probability distributions. We adopt transfer learning to apply $N$ on ${\mathcal N}$ and ${\mathcal M}$ separately and efficiently, combining the results to obtain an online learner. Second, we formulate a specific loss function more suited for deep learning and use iterative training to solve it in the online setting. On publicly available datasets, our online learners achieve a $99.3\%$ improvement on false-positive rates compared to the baseline detection methods. In the offline setting, our approaches are competitive with classifiers trained on labeled data.

Related papers

• Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization [65.8915778873691]
  conditional distributions is a central problem in machine learning. We propose a new learning paradigm that integrates both paired and unpaired data. Our approach also connects intriguingly with inverse entropic optimal transport (OT)
  arXiv  Detail & Related papers  (2024-10-03T16:12:59Z)
• Predict And Prevent DDOS Attacks Using Machine Learning and Statistical Algorithms [0.0]
  This study uses several machine learning and statistical models to detect DDoS attacks from traces of traffic flow. The XGboost machine learning model provided the best detection accuracy of (99.9999%) after applying the SMOTE approach to the target class.
  arXiv  Detail & Related papers  (2023-08-30T00:03:32Z)
• Optimal Best-Arm Identification in Bandits with Access to Offline Data [27.365122983434887]
  We consider combining offline data with online learning, an area less studied but obvious practical importance. We develop algorithms that match the lower bound on sample complexity when $delta$ is small. Our algorithms are computationally efficient with an average per-sample acquisition cost of $tildeO(K)$, and rely on a careful characterization of the optimality conditions of the lower bound problem.
  arXiv  Detail & Related papers  (2023-06-15T11:12:35Z)
• Efficient Online Reinforcement Learning with Offline Data [78.92501185886569]
  We show that we can simply apply existing off-policy methods to leverage offline data when learning online. We extensively ablate these design choices, demonstrating the key factors that most affect performance. We see that correct application of these simple recommendations can provide a $mathbf2.5times$ improvement over existing approaches.
  arXiv  Detail & Related papers  (2023-02-06T17:30:22Z)
• APP: Anytime Progressive Pruning [104.36308667437397]
  We propose a novel way of training a neural network with a target sparsity in a particular case of online learning: the anytime learning at macroscale paradigm (ALMA) The proposed approach significantly outperforms the baseline dense and Anytime OSP models across multiple architectures and datasets under short, moderate, and long-sequence training.
  arXiv  Detail & Related papers  (2022-04-04T16:38:55Z)
• Modern Cybersecurity Solution using Supervised Machine Learning [0.456877715768796]
  Traditional Firewall and Intrusion Detection system fails to detect new attacks, zero-day attacks, and traffic patterns that do not match with configured rules. We used Netflow datasets to extract features after applying data analysis. Our experiments focus on how efficient machine learning algorithms can detect Bot traffic, Malware traffic, and background traffic.
  arXiv  Detail & Related papers  (2021-09-15T22:03:50Z)
• Online Adversarial Attacks [57.448101834579624]
  We formalize the online adversarial attack problem, emphasizing two key elements found in real-world use-cases. We first rigorously analyze a deterministic variant of the online threat model. We then propose algoname, a simple yet practical algorithm yielding a provably better competitive ratio for $k=2$ over the current best single threshold algorithm.
  arXiv  Detail & Related papers  (2021-03-02T20:36:04Z)
• Online Apprenticeship Learning [58.45089581278177]
  In Apprenticeship Learning (AL), we are given a Markov Decision Process (MDP) without access to the cost function. The goal is to find a policy that matches the expert's performance on some predefined set of cost functions. We show that the OAL problem can be effectively solved by combining two mirror descent based no-regret algorithms.
  arXiv  Detail & Related papers  (2021-02-13T12:57:51Z)
• Online Markov Decision Processes with Aggregate Bandit Feedback [74.85532145498742]
  We study a novel variant of online finite-horizon Markov Decision Processes with adversarially changing loss functions and initially unknown dynamics. In each episode, the learner suffers the loss accumulated along the trajectory realized by the policy chosen for the episode, and observes aggregate bandit feedback. Our main result is a computationally efficient algorithm with $O(sqrtK)$ regret for this setting, where $K$ is the number of episodes.
  arXiv  Detail & Related papers  (2021-01-31T16:49:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.