Assessing Generalisation Capability of Machine Learning Models for Intrusion Detection

• URL: http://arxiv.org/abs/2605.04407v1
• Date: Wed, 06 May 2026 01:59:09 GMT
• Title: Assessing Generalisation Capability of Machine Learning Models for Intrusion Detection
• Abstract summary: This study investigates the generalisation capability of supervised machine learning models for intrusion detection using UNSW-NB15 and TON_IoT.<n> Random Forest, Logistic Regression, and Naive Bayes were evaluated under same-dataset and cross-dataset settings.<n>Results reveal a significant generalisation gap in intrusion detection.
• Score: 12.976685413458243
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The growth of networked and IoT systems has intensified cyber-security threats and exposed the limits of traditional signature-based intrusion detection. Although machine-learning-based intrusion detection systems often report strong benchmark performance, high ac- curacy within a single dataset does not necessarily guarantee reliable performance in unseen network environments. This study investigates the generalisation capability of supervised machine learning models for intrusion detection using UNSW-NB15 and TON_IoT. Random Forest, Logistic Regression, and Naive Bayes were evaluated under same-dataset and cross-dataset settings. Random Forest achieved the strongest same dataset performance, with 95.08% accuracy on UNSW-NB15 and 99.79% on TON_IoT, but performance dropped sharply in cross-dataset testing. When trained on UNSW-NB15 and tested on TON_IoT or vice versa, below 40% accuracy. These results reveal a significant generalisation gap in intrusion detection. We connect this challenge to affective computing and human-centric AI, where behavioural signal analysis, anomaly detection, domain shift, and context-sensitive modelling are also central. This framing highlights the need for adaptive, generalisable cyber-security models that can operate across changing network and IoT environments.

Related papers

• Robustness Analysis of Machine Learning Models for IoT Intrusion Detection Under Data Poisoning Attacks [0.0]
  This study evaluates the susceptibility of four widely used classifiers, Random Forest, Gradient Boosting Machine, Logistic Regression, and Deep Neural Network models, against multiple poisoning strategies.<n>Results show that while ensemble-based models exhibit comparatively stable performance, Logistic Regression and Deep Neural Networks suffer degradation of up to 40% under label manipulation and outlier-based attacks.<n>Findings highlight the need for adversarially robust training, continuous anomaly monitoring, and feature-level validation within operational Network Intrusion Detection Systems.
  arXiv  Detail & Related papers  (2026-04-15T21:36:44Z)
• Understanding AI Methods for Intrusion Detection and Cryptographic Leakage [42.8384136157456]
  We evaluate how machine learning techniques can detect malicious network activity and identify potential information leakage in cryptographic implementations.<n>Our results demonstrate that AI models can achieve near-perfect detection accuracy within stable network environment.<n>We also observed that learned models identify patterns consistent with side-channel leakage, suggesting that AI can assist in uncovering implementation-level vulnerabilities.
  arXiv  Detail & Related papers  (2026-03-26T18:42:57Z)
• Neurosymbolic Learning for Advanced Persistent Threat Detection under Extreme Class Imbalance [29.991707658663188]
  This paper proposes a neurosymbolic architecture that integrates an optimized BERT model with logic tensor networks (LTN) for explainable APT detection in wireless IoT networks.<n>Results show that neurosymbolic learning enables high-performance, interpretable, and operationally viable APT detection for IoT network monitoring architectures.
  arXiv  Detail & Related papers  (2026-02-28T04:25:50Z)
• Exploring Robust Intrusion Detection: A Benchmark Study of Feature Transferability in IoT Botnet Attack Detection [46.724153214302184]
  Cross-domain intrusion detection remains a critical challenge due to significant variability in network traffic characteristics and feature distributions across environments.<n>This study evaluates the transferability of three widely used flow-based feature sets (Argus, Zeek and CICFlowMeter) across four widely used datasets representing heterogeneous IoT and Industrial IoT network conditions.
  arXiv  Detail & Related papers  (2026-02-27T10:16:28Z)
• Multi-Agent Collaborative Intrusion Detection for Low-Altitude Economy IoT: An LLM-Enhanced Agentic AI Framework [60.72591149679355]
  The rapid expansion of low-altitude economy Internet of Things (LAE-IoT) networks has created unprecedented security challenges.<n>Traditional intrusion detection systems fail to tackle the unique characteristics of aerial IoT environments.<n>We introduce a large language model (LLM)-enabled agentic AI framework for enhancing intrusion detection in LAE-IoT networks.
  arXiv  Detail & Related papers  (2026-01-25T12:47:25Z)
• CST-AFNet: A dual attention-based deep learning framework for intrusion detection in IoT networks [0.2180739748940442]
  This research presents CST AFNet, a novel dual attention based deep learning framework for robust intrusion detection in IoT networks.<n>The proposed model achieves outstanding accuracy for both 15 attack types and benign traffic.<n>The findings confirm that CST AFNet is a powerful and scalable solution for real time cyber threat detection in complex IoT and IIoT environments.
  arXiv  Detail & Related papers  (2025-10-03T04:36:19Z)
• REAL-IoT: Characterizing GNN Intrusion Detection Robustness under Practical Adversarial Attack [5.881825061973424]
  Graph Neural Network (GNN)-based network intrusion detection systems (NIDS) are often evaluated on single datasets.<n>We propose textbfREAL-IoT, a comprehensive framework for robustness evaluation of GNN-based NIDS in IoT environments.
  arXiv  Detail & Related papers  (2025-07-14T22:10:08Z)
• Self-Supervised Transformer-based Contrastive Learning for Intrusion Detection Systems [1.1265248232450553]
  This paper proposes a self-supervised contrastive learning approach for generalizable intrusion detection on raw packet sequences.<n>Our framework exhibits better performance in comparison to existing NetFlow self-supervised methods.<n>Our model provides a strong baseline for supervised intrusion detection with limited labeled data.
  arXiv  Detail & Related papers  (2025-05-12T13:42:00Z)
• WeiDetect: Weibull Distribution-Based Defense against Poisoning Attacks in Federated Learning for Network Intrusion Detection Systems [23.03944479383518]
  We propose WeiDetect, a two-phase, server-side defense mechanism for FL-based NIDS that detects malicious participants.<n>We conducted experiments to evaluate the effectiveness of our approach in diverse attack settings.<n>Our findings highlight that WeiDetect outperforms state-of-the-art defense approaches.
  arXiv  Detail & Related papers  (2025-04-06T05:31:24Z)
• FedMSE: Semi-supervised federated learning approach for IoT network intrusion detection [0.0]
  The rise of IoT has expanded the cyber attack surface, making traditional centralized machine learning methods insufficient due to concerns about data availability, computational resources, transfer costs, and especially privacy preservation.<n>A semi-supervised federated learning model was developed to overcome these issues, combining the Shrink Autoencoder and Centroid one-class classifier (SAE-CEN)<n>This approach enhances the performance of intrusion detection by effectively representing normal network data and accurately identifying anomalies in the decentralized strategy.
  arXiv  Detail & Related papers  (2024-10-18T02:23:57Z)
• Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction [6.374540518226326]
  Intrusion Detection Systems (IDS) play a critical role in protecting interconnected networks by detecting malicious actors and activities. This paper introduces a novel ML-based network intrusion detection model that uses Random Oversampling (RO) to address data imbalance and Stacking Feature Embedding (PCA) for dimension reduction. Using the CIC-IDS 2017 dataset, DT, RF, and ET models reach 99.99% accuracy, while DT and RF models obtain 99.94% accuracy on CIC-IDS 2018 dataset.
  arXiv  Detail & Related papers  (2024-01-22T05:49:41Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.