Evaluation of AutoML Frameworks for IDS under Imbalanced Data Conditions of the NSL-KDD Dataset

• URL: http://arxiv.org/abs/2606.12611v1
• Date: Wed, 10 Jun 2026 19:08:25 GMT
• Title: Evaluation of AutoML Frameworks for IDS under Imbalanced Data Conditions of the NSL-KDD Dataset
• Abstract summary: This work investigates the impact of severe class imbalance on the performance of automated machine learning (AutoML) frameworks for network intrusion detection.<n>We preserve the original five-class distribution, including highly underrepresented attacks such as R2L and U2R.<n>Nine open-source AutoML frameworks were analyzed under a unified and reproducible experimental protocol.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work investigates the impact of severe class imbalance on the performance of automated machine learning (AutoML) frameworks for multiclass network intrusion detection using the NSL-KDD dataset. Unlike previous studies that simplify the problem through binary classification or minority-class removal, we preserve the original five-class distribution, including highly underrepresented attacks such as R2L and U2R, enabling a realistic evaluation of imbalance-sensitive learning behavior. Nine open-source AutoML frameworks were analyzed under a unified and reproducible experimental protocol, considering differences in architectural design, ensemble strategies, validation procedures, hyperparameter optimization, and imbalance-handling mechanisms. The results demonstrate that frameworks incorporating ensemble learning and imbalance-aware optimization achieve better minority-class discrimination. PyCaret obtained the best overall performance, reaching 66\% macro-F1, followed by AutoGluon with 55\%, whereas frameworks lacking native balancing support exhibited significant degradation in minority-class detection capability. The analysis further shows that accuracy-oriented optimization alone is insufficient for highly imbalanced IDS scenarios, since high-weighted metrics may coexist with poor generalization on rare attack categories. As a contribution, this work establishes a standardized benchmark for AutoML-based intrusion detection under severe multiclass imbalance, highlighting current architectural limitations and the need for native integration of imbalance-aware optimization, resampling, and stratified evaluation strategies into automated learning pipelines. The source code is publicly available.

Related papers

• Hybrid Imbalanced Regression Through Unified Data-Level and Algorithm-Level Balancing [1.3700362496838856]
  Imbalanced learning is a critical challenge in machine learning, where underrepresented target values can bias models and degrade prediction performance on rare but important cases.<n>We propose a unified hybrid framework that integrates both data- and algorithm-level balancing strategies into a regressor-agnostic pipeline.<n> Experimental results on benchmark datasets demonstrate that the proposed framework consistently improves predictive performance compared to standalone regressors and existing imbalanced regression approaches.
  arXiv  Detail & Related papers  (2026-05-31T13:13:10Z)
• Optimized Deferral for Imbalanced Settings [52.48138985215643]
  Learning algorithms can be significantly improved by routing complex or uncertain inputs to specialized experts.<n>This approach, known as learning to defer, is essential in domains like natural language generation, medical diagnosis, and computer vision.<n>We present a comprehensive study of two-stage learning to defer in expert imbalance settings.
  arXiv  Detail & Related papers  (2026-04-30T11:15:37Z)
• Model-Agnostic Meta Learning for Class Imbalance Adaptation [3.2488051786523813]
  Class imbalance is a widespread challenge in NLP tasks, hindering robust performance across diverse domains and applications.<n>We introduce Hardness-Aware Meta-Resample (HAMR), a unified framework that adaptively addresses both class imbalance and data difficulty.
  arXiv  Detail & Related papers  (2026-04-20T19:07:56Z)
• AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration [49.050688065974605]
  Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models.<n>We propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution.
  arXiv  Detail & Related papers  (2026-04-01T15:17:45Z)
• White-Basilisk: A Hybrid Model for Code Vulnerability Detection [45.03594130075282]
  We introduce White-Basilisk, a novel approach to vulnerability detection that demonstrates superior performance.<n>White-Basilisk achieves results in vulnerability detection tasks with a parameter count of only 200M.<n>This research establishes new benchmarks in code security and provides empirical evidence that compact, efficiently designed models can outperform larger counterparts in specialized tasks.
  arXiv  Detail & Related papers  (2025-07-11T12:39:25Z)
• Taming Polysemanticity in LLMs: Provable Feature Recovery via Sparse Autoencoders [50.52694757593443]
  Existing SAE training algorithms often lack rigorous mathematical guarantees and suffer from practical limitations.<n>We first propose a novel statistical framework for the feature recovery problem, which includes a new notion of feature identifiability.<n>We introduce a new SAE training algorithm based on bias adaptation'', a technique that adaptively adjusts neural network bias parameters to ensure appropriate activation sparsity.
  arXiv  Detail & Related papers  (2025-06-16T20:58:05Z)
• Synthetic Code Surgery: Repairing Bugs and Vulnerabilities with LLMs and Synthetic Data [0.0]
  This paper presents a novel methodology for enhancing Automated Program Repair (APR) through synthetic data generation utilizing Large Language Models (LLMs)<n>The proposed approach addresses this limitation through a two-phase process: a synthetic sample generation followed by a rigorous quality assessment.<n> Experimental evaluation on the VulRepair test set dataset showed statistically significant improvements in Perfect Prediction rates.
  arXiv  Detail & Related papers  (2025-05-12T09:14:20Z)
• FILM: Framework for Imbalanced Learning Machines based on a new unbiased performance measure and a new ensemble-based technique [37.94431794242543]
  This research addresses the challenges of handling unbalanced datasets for binary classification tasks.<n>Standard evaluation metrics are often biased by the disproportionate representation of the minority class.<n>We propose a novel metric, the Unbiased Integration Coefficients, which exhibits significantly reduced bias.
  arXiv  Detail & Related papers  (2025-03-06T12:15:56Z)
• Adaptive Cyber-Attack Detection in IIoT Using Attention-Based LSTM-CNN Models [0.23408308015481666]
  This study presents the development and evaluation of an advanced Intrusion detection (IDS) based on a hybrid LSTM-convolution neural network (CNN)-Attention architecture.<n>The research focuses on two key classification tasks: binary and multi-class classification.<n>In binary classification, the model achieved near-perfect accuracy, while in multi-class classification, it maintained a high accuracy level (99.04%), effectively categorizing different attack types with a loss value of 0.0220%.
  arXiv  Detail & Related papers  (2025-01-21T20:52:23Z)
• Group & Reweight: A Novel Cost-Sensitive Approach to Mitigating Class Imbalance in Network Traffic Classification [8.38878965125967]
  This paper focuses on network traffic classification in the presence of severe class imbalance.<n>We design a group & reweight strategy for alleviating class imbalance.<n>Results show that our approach can not only suppress the negative effect of class imbalance but also improve the comprehensive performance in prediction.
  arXiv  Detail & Related papers  (2024-09-28T02:45:14Z)
• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• Learning with Multiclass AUC: Theory and Algorithms [141.63211412386283]
  Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics.
  arXiv  Detail & Related papers  (2021-07-28T05:18:10Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.