Robust Anomaly Detection in Network Traffic: Evaluating Machine Learning Models on CICIDS2017

• URL: http://arxiv.org/abs/2506.19877v1
• Date: Mon, 23 Jun 2025 15:31:10 GMT
• Title: Robust Anomaly Detection in Network Traffic: Evaluating Machine Learning Models on CICIDS2017
• Authors: Zhaoyang Xu, Yunbo Liu,
• Abstract summary: We present a comparison of four representative models on the CICIDS 2017 dataset.<n>Supervised and CNN achieve near-perfect accuracy on familiar attacks but suffer drastic recall drops on novel attacks.<n>Unsupervised LOF attains moderate overall accuracy and high recall on unknown threats at the cost of elevated false alarms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying suitable machine learning paradigms for intrusion detection remains critical for building effective and generalizable security solutions. In this study, we present a controlled comparison of four representative models - Multi-Layer Perceptron (MLP), 1D Convolutional Neural Network (CNN), One-Class Support Vector Machine (OCSVM) and Local Outlier Factor (LOF) - on the CICIDS2017 dataset under two scenarios: detecting known attack types and generalizing to previously unseen threats. Our results show that supervised MLP and CNN achieve near-perfect accuracy on familiar attacks but suffer drastic recall drops on novel attacks. Unsupervised LOF attains moderate overall accuracy and high recall on unknown threats at the cost of elevated false alarms, while boundary-based OCSVM balances precision and recall best, demonstrating robust detection across both scenarios. These findings offer practical guidance for selecting IDS models in dynamic network environments.

Related papers

• Adversarial Attacks on Deep Learning-Based False Data Injection Detection in Differential Relays [3.4061238650474666]
  This paper demonstrates that adversarial attacks, carefully crafted FDIAs, can evade existing Deep Learning-based Schemes (DLSs) used for False Data Injection Attacks (FDIAs) in smart grids.<n>We propose a novel adversarial attack framework, utilizing the Fast Gradient Sign Method, which exploits DLS vulnerabilities.<n>Our results highlight the significant threat posed by adversarial attacks to DLS-based FDIA detection, underscore the necessity for robust cybersecurity measures in smart grids, and demonstrate the effectiveness of adversarial training in enhancing model robustness against adversarial FDIAs.
  arXiv  Detail & Related papers  (2025-06-24T04:22:26Z)
• Robustness of Selected Learning Models under Label-Flipping Attack [1.3812010983144798]
  We compare traditional machine learning and deep learning models trained on a malware dataset when subjected to adversarial attack based on label-flipping.<n>We find wide variation in the robustness of the models tested to adversarial attack, with our model achieving the best combination of initial accuracy and robustness.
  arXiv  Detail & Related papers  (2025-01-21T22:00:54Z)
• usfAD Based Effective Unknown Attack Detection Focused IDS Framework [3.560574387648533]
  Internet of Things (IoT) and Industrial Internet of Things (IIoT) have led to an increasing range of cyber threats. For more than a decade, researchers have delved into supervised machine learning techniques to develop Intrusion Detection System (IDS) IDS trained and tested on known datasets fails in detecting zero-day or unknown attacks. We propose two strategies for semi-supervised learning based IDS where training samples of attacks are not required.
  arXiv  Detail & Related papers  (2024-03-17T11:49:57Z)
• Securing Graph Neural Networks in MLaaS: A Comprehensive Realization of Query-based Integrity Verification [68.86863899919358]
  We introduce a groundbreaking approach to protect GNN models in Machine Learning from model-centric attacks. Our approach includes a comprehensive verification schema for GNN's integrity, taking into account both transductive and inductive GNNs. We propose a query-based verification technique, fortified with innovative node fingerprint generation algorithms.
  arXiv  Detail & Related papers  (2023-12-13T03:17:05Z)
• Unveiling the potential of Graph Neural Networks for robust Intrusion Detection [2.21481607673149]
  We propose a novel Graph Neural Network (GNN) model to learn flow patterns of attacks structured as graphs. Our model is able to maintain the same level of accuracy as in previous experiments, while state-of-the-art ML techniques degrade up to 50% their accuracy (F1-score) under adversarial attacks.
  arXiv  Detail & Related papers  (2021-07-30T16:56:39Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Anomaly Detection in Cybersecurity: Unsupervised, Graph-Based and Supervised Learning Methods in Adversarial Environments [63.942632088208505]
  Inherent to today's operating environment is the practice of adversarial machine learning. In this work, we examine the feasibility of unsupervised learning and graph-based methods for anomaly detection. We incorporate a realistic adversarial training mechanism when training our supervised models to enable strong classification performance in adversarial environments.
  arXiv  Detail & Related papers  (2021-05-14T10:05:10Z)
• How Robust are Randomized Smoothing based Defenses to Data Poisoning? [66.80663779176979]
  We present a previously unrecognized threat to robust machine learning models that highlights the importance of training-data quality. We propose a novel bilevel optimization-based data poisoning attack that degrades the robustness guarantees of certifiably robust classifiers. Our attack is effective even when the victim trains the models from scratch using state-of-the-art robust training methods.
  arXiv  Detail & Related papers  (2020-12-02T15:30:21Z)
• Bayesian Optimization with Machine Learning Algorithms Towards Anomaly Detection [66.05992706105224]
  In this paper, an effective anomaly detection framework is proposed utilizing Bayesian Optimization technique. The performance of the considered algorithms is evaluated using the ISCX 2012 dataset. Experimental results show the effectiveness of the proposed framework in term of accuracy rate, precision, low-false alarm rate, and recall.
  arXiv  Detail & Related papers  (2020-08-05T19:29:35Z)
• Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification [78.51092318750102]
  This work proposes to defend ASV systems against adversarial attacks with a separate detection network. A VGG-like binary classification detector is introduced and demonstrated to be effective on detecting adversarial samples.
  arXiv  Detail & Related papers  (2020-06-11T04:31:56Z)
• RAB: Provable Robustness Against Backdoor Attacks [20.702977915926787]
  We focus on certifying the machine learning model robustness against general threat models, especially backdoor attacks. We propose the first robust training process, RAB, to smooth the trained model and certify its robustness against backdoor attacks. We conduct comprehensive experiments for different machine learning (ML) models and provide the first benchmark for certified robustness against backdoor attacks.
  arXiv  Detail & Related papers  (2020-03-19T17:05:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.