Continual Learning with Strategic Selection and Forgetting for Network Intrusion Detection

• URL: http://arxiv.org/abs/2412.16264v3
• Date: Fri, 14 Feb 2025 12:15:37 GMT
• Title: Continual Learning with Strategic Selection and Forgetting for Network Intrusion Detection
• Authors: Xinchen Zhang, Running Zhao, Zhihan Jiang, Handi Chen, Yulong Ding, Edith C. H. Ngai, Shuang-Hua Yang,
• Abstract summary: Intrusion Detection Systems (IDS) are crucial for safeguarding digital infrastructure.<n>In this paper, we propose SSF (Strategic Selection and Forgetting), a novel continual learning method for IDS.<n>Our approach features a strategic sample selection algorithm to select representative new samples and a strategic forgetting mechanism to drop outdated samples.
• Score: 6.3399691183255165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intrusion Detection Systems (IDS) are crucial for safeguarding digital infrastructure. In dynamic network environments, both threat landscapes and normal operational behaviors are constantly changing, resulting in concept drift. While continuous learning mitigates the adverse effects of concept drift, insufficient attention to drift patterns and excessive preservation of outdated knowledge can still hinder the IDS's adaptability. In this paper, we propose SSF (Strategic Selection and Forgetting), a novel continual learning method for IDS, providing continuous model updates with a constantly refreshed memory buffer. Our approach features a strategic sample selection algorithm to select representative new samples and a strategic forgetting mechanism to drop outdated samples. The proposed strategic sample selection algorithm prioritizes new samples that cause the `drifted' pattern, enabling the model to better understand the evolving landscape. Additionally, we introduce strategic forgetting upon detecting significant drift by discarding outdated samples to free up memory, allowing the incorporation of more recent data. SSF captures evolving patterns effectively and ensures the model is aligned with the change of data patterns, significantly enhancing the IDS's adaptability to concept drift. The state-of-the-art performance of SSF on NSL-KDD and UNSW-NB15 datasets demonstrates its superior adaptability to concept drift for network intrusion detection. The code is released at https://github.com/xinchen930/SSF-Strategic-Selection-and-Forgetting.

Related papers

• AiGAS-dEVL-RC: An Adaptive Growing Neural Gas Model for Recurrently Drifting Unsupervised Data Streams [6.7236795813629]
  This work introduces a novel method based on the Growing Neural Gas (GNG) algorithm to handle abrupt recurrent drifts. The proposed approach maintains a compact yet informative memory structure, allowing it to efficiently store and retrieve knowledge of past or recurring concepts. Unlike other techniques that fail to leverage recurring knowledge, our proposed approach is proven to be a robust and efficient online learning solution for unsupervised drifting data flows.
  arXiv  Detail & Related papers  (2025-04-08T07:42:50Z)
• Generative Active Adaptation for Drifting and Imbalanced Network Intrusion Detection [15.146203784334086]
  We propose a generative active adaptation framework that minimizes labeling effort while enhancing model robustness. We evaluate our end-to-end framework on both simulated IDS data and a real-world ISP dataset. Our framework effectively enhances rare attack detection while reducing labeling costs, making it a scalable and adaptive solution for real-world intrusion detection.
  arXiv  Detail & Related papers  (2025-03-04T21:49:42Z)
• Unseen Attack Detection in Software-Defined Networking Using a BERT-Based Large Language Model [5.062869359266078]
  We introduce a novel approach that leverages Natural Language Processing (NLP) and the pre trained BERT base model to enhance attack detection in Software defined networking (SDN)<n>Our approach transforms network flow data into a format interpretable by language models, allowing BERT to capture intricate patterns and relationships within network traffic.<n>Our approach is specifically designed to detect previously unseen attacks, offering a solution for identifying threats that the model was not explicitly trained on.
  arXiv  Detail & Related papers  (2024-12-09T06:27:20Z)
• Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture [58.60915132222421]
  We introduce an approach that is both general and parameter-efficient for face forgery detection. We design a forgery-style mixture formulation that augments the diversity of forgery source domains. We show that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters.
  arXiv  Detail & Related papers  (2024-08-23T01:53:36Z)
• Neuromimetic metaplasticity for adaptive continual learning [2.1749194587826026]
  We propose a metaplasticity model inspired by human working memory to achieve catastrophic forgetting-free continual learning. A key aspect of our approach involves implementing distinct types of synapses from stable to flexible, and randomly intermixing them to train synaptic connections with different degrees of flexibility. The model achieved a balanced tradeoff between memory capacity and performance without requiring additional training or structural modifications.
  arXiv  Detail & Related papers  (2024-07-09T12:21:35Z)
• An incremental hybrid adaptive network-based IDS in Software Defined Networks to detect stealth attacks [0.0]
  Advanced Persistent Threats (APTs) are a type of attack that implement a wide range of strategies to evade detection. Machine Learning (ML) techniques in Intrusion Detection Systems (IDSs) is widely used to detect such attacks but has a challenge when the data distribution changes. An incremental hybrid adaptive Network Intrusion Detection System (NIDS) is proposed to tackle the issue of concept drift in SDN.
  arXiv  Detail & Related papers  (2024-04-01T13:33:40Z)
• METER: A Dynamic Concept Adaptation Framework for Online Anomaly Detection [25.022228143354123]
  Real-time analytics and decision-making require online anomaly detection to handle drifts in data streams efficiently and effectively. Existing approaches are often constrained by their limited detection capacity and slow adaptation to evolving data streams. We introduce METER, a novel dynamic concept adaptation framework that introduces a new paradigm for OAD.
  arXiv  Detail & Related papers  (2023-12-28T05:09:31Z)
• Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective [64.04617968947697]
  We introduce a novel data-model co-design perspective: to promote superior weight sparsity. Specifically, customized Visual Prompts are mounted to upgrade neural Network sparsification in our proposed VPNs framework.
  arXiv  Detail & Related papers  (2023-12-03T13:50:24Z)
• FLARE: Detection and Mitigation of Concept Drift for Federated Learning based IoT Deployments [2.7776688429637466]
  FLARE is a lightweight dual-scheduler FL framework that conditionally transfers training data and deploys models between edge and sensor endpoints. We show that FLARE can significantly reduce the amount of data exchanged between edge and sensor nodes compared to fixed-interval scheduling methods. It can successfully detect concept drift reactively with at least a 16x reduction in latency.
  arXiv  Detail & Related papers  (2023-05-15T10:09:07Z)
• FOSTER: Feature Boosting and Compression for Class-Incremental Learning [52.603520403933985]
  Deep neural networks suffer from catastrophic forgetting when learning new categories. We propose a novel two-stage learning paradigm FOSTER, empowering the model to learn new categories adaptively.
  arXiv  Detail & Related papers  (2022-04-10T11:38:33Z)
• Learning Fast and Slow for Online Time Series Forecasting [76.50127663309604]
  Fast and Slow learning Networks (FSNet) is a holistic framework for online time-series forecasting. FSNet balances fast adaptation to recent changes and retrieving similar old knowledge. Our code will be made publicly available.
  arXiv  Detail & Related papers  (2022-02-23T18:23:07Z)
• Context-Preserving Instance-Level Augmentation and Deformable Convolution Networks for SAR Ship Detection [50.53262868498824]
  Shape deformation of targets in SAR image due to random orientation and partial information loss is an essential challenge in SAR ship detection. We propose a data augmentation method to train a deep network that is robust to partial information loss within the targets.
  arXiv  Detail & Related papers  (2022-02-14T07:01:01Z)
• Learning to Detect: A Data-driven Approach for Network Intrusion Detection [17.288512506016612]
  We perform a comprehensive study on NSL-KDD, a network traffic dataset, by visualizing patterns and employing different learning-based models to detect cyber attacks. Unlike previous shallow learning and deep learning models that use the single learning model approach for intrusion detection, we adopt a hierarchy strategy. We demonstrate the advantage of the unsupervised representation learning model in binary intrusion detection tasks.
  arXiv  Detail & Related papers  (2021-08-18T21:19:26Z)
• On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
  Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time. We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
  arXiv  Detail & Related papers  (2020-07-16T18:39:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.