Bridging the gap to real-world for network intrusion detection systems with data-centric approach

• URL: http://arxiv.org/abs/2110.13655v1
• Date: Mon, 25 Oct 2021 04:50:12 GMT
• Title: Bridging the gap to real-world for network intrusion detection systems with data-centric approach
• Authors: Gustavo de Carvalho Bertoli, Louren\c{c}o Alves Pereira Junior, Filipe Alves Neto Verri, Aldri Luiz dos Santos, Osamu Saotome
• Abstract summary: This paper presents a systematic data-centric approach to address the current limitations of NIDS research. It generates NIDS datasets composed of the most recent network traffic and attacks, with the labeling process integrated by design.
• Score: 1.4699455652461724
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most research using machine learning (ML) for network intrusion detection systems (NIDS) uses well-established datasets such as KDD-CUP99, NSL-KDD, UNSW-NB15, and CICIDS-2017. In this context, the possibilities of machine learning techniques are explored, aiming for metrics improvements compared to the published baselines (model-centric approach). However, those datasets present some limitations as aging that make it unfeasible to transpose those ML-based solutions to real-world applications. This paper presents a systematic data-centric approach to address the current limitations of NIDS research, specifically the datasets. This approach generates NIDS datasets composed of the most recent network traffic and attacks, with the labeling process integrated by design.

Related papers

• Always be Pre-Training: Representation Learning for Network Intrusion Detection with GNNs [6.589041710104928]
  Graph neural network-based network intrusion detection systems have recently demonstrated state-of-the-art performance on benchmark datasets. These methods suffer from a reliance on target encoding for data pre-processing, limiting widespread adoption due to the associated need for annotated labels. We propose a solution involving in-context pre-training and the utilization of dense representations for categorical features to jointly overcome the label-dependency limitation.
  arXiv  Detail & Related papers  (2024-02-29T09:40:07Z)
• On the Cross-Dataset Generalization of Machine Learning for Network Intrusion Detection [50.38534263407915]
  Network Intrusion Detection Systems (NIDS) are a fundamental tool in cybersecurity. Their ability to generalize across diverse networks is a critical factor in their effectiveness and a prerequisite for real-world applications. In this study, we conduct a comprehensive analysis on the generalization of machine-learning-based NIDS through an extensive experimentation in a cross-dataset framework.
  arXiv  Detail & Related papers  (2024-02-15T14:39:58Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• Hybridization of Capsule and LSTM Networks for unsupervised anomaly detection on multivariate data [0.0]
  This paper introduces a novel NN architecture which hybridises the Long-Short-Term-Memory (LSTM) and Capsule Networks into a single network. The proposed method uses an unsupervised learning technique to overcome the issues with finding large volumes of labelled training data.
  arXiv  Detail & Related papers  (2022-02-11T10:33:53Z)
• Improving the Reliability of Network Intrusion Detection Systems through Dataset Integration [0.20646127669654826]
  This work presents Reliable-NIDS (R-NIDS), a novel methodology for Machine Learning (ML) based Network Intrusion Detection Systems (NIDSs) R-NIDS allows ML models to work on integrated datasets, empowering the learning process with diverse information from different datasets. In this work we propose to build two well-known ML models based on the information of three of the most common datasets in the literature for NIDS evaluation.
  arXiv  Detail & Related papers  (2021-12-02T09:30:18Z)
• Feature Extraction for Machine Learning-based Intrusion Detection in IoT Networks [6.6147550436077776]
  This paper aims to discover whether Feature Reduction (FR) and Machine Learning (ML) techniques can be generalised across various datasets. The detection accuracy of three Feature Extraction (FE) algorithms; Principal Component Analysis (PCA), Auto-encoder (AE), and Linear Discriminant Analysis (LDA) is evaluated.
  arXiv  Detail & Related papers  (2021-08-28T23:52:18Z)
• Evaluating Federated Learning for Intrusion Detection in Internet of Things: Review and Challenges [0.0]
  Federated Learning (FL) has attracted a significant interest in different sectors, including healthcare and transport systems. We evaluate a FL-enabled IDS approach based on a multiclass classifier considering different data distributions for the detection of different attacks in an IoT scenario. We identify a set of challenges and future directions based on the existing literature and the analysis of our evaluation results.
  arXiv  Detail & Related papers  (2021-08-02T15:22:05Z)
• Data Mining with Big Data in Intrusion Detection Systems: A Systematic Literature Review [68.15472610671748]
  Cloud computing has become a powerful and indispensable technology for complex, high performance and scalable computation. The rapid rate and volume of data creation has begun to pose significant challenges for data management and security. The design and deployment of intrusion detection systems (IDS) in the big data setting has, therefore, become a topic of importance.
  arXiv  Detail & Related papers  (2020-05-23T20:57:12Z)
• Survey of Network Intrusion Detection Methods from the Perspective of the Knowledge Discovery in Databases Process [63.75363908696257]
  We review the methods that have been applied to network data with the purpose of developing an intrusion detector. We discuss the techniques used for the capture, preparation and transformation of the data, as well as, the data mining and evaluation methods. As a result of this literature review, we investigate some open issues which will need to be considered for further research in the area of network security.
  arXiv  Detail & Related papers  (2020-01-27T11:21:05Z)
• Deep Learning based Pedestrian Inertial Navigation: Methods, Dataset and On-Device Inference [49.88536971774444]
  Inertial measurements units (IMUs) are small, cheap, energy efficient, and widely employed in smart devices and mobile robots. Exploiting inertial data for accurate and reliable pedestrian navigation supports is a key component for emerging Internet-of-Things applications and services. We present and release the Oxford Inertial Odometry dataset (OxIOD), a first-of-its-kind public dataset for deep learning based inertial navigation research.
  arXiv  Detail & Related papers  (2020-01-13T04:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.