A Three-Axis Stress Test of LLM vs Classical ML for Network Intrusion Detection under Distribution Shift and Adversarial Evasion

• URL: http://arxiv.org/abs/2609.13511v1
• Date: Fri, 11 Sep 2026 20:23:30 GMT
• Title: A Three-Axis Stress Test of LLM vs Classical ML for Network Intrusion Detection under Distribution Shift and Adversarial Evasion
• Abstract summary: We evaluate XGBoost and RoBERTa-LoRA on two independently collected NetFlow v2 networks across three axes.<n>XGBoost wins decisively under cross-dataset distribution shift, by 15 points of F1 and 25 points of balanced accuracy.<n>RoBERTa-LoRA wins decisively under adversarial evasion, by roughly 17 points of F1 at a representative mid-range perturbation strength.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models are increasingly benchmarked against classical machine learning for network intrusion detection (NIDS), almost always using same-dataset evaluation, and that protocol turns out to be incomplete. Evaluating XGBoost and RoBERTa-LoRA on two independently collected NetFlow v2 networks across three axes (same-dataset performance, cross-dataset transfer, and adversarial evasion) reveals no universal winner. The two models are statistically tied same-dataset. XGBoost wins decisively under cross-dataset distribution shift, by 15 points of F1 and 25 points of balanced accuracy; on the target network RoBERTa-LoRA's false positive rate reaches 0.78, leaving it barely above chance despite a superficially moderate F1. RoBERTa-LoRA wins decisively under adversarial evasion, by roughly 17 points of F1 at a representative mid-range perturbation strength, while both models hold false positive rates below 0.01 throughout. The model an evaluator would recommend therefore depends entirely on which axis is tested, not on same-dataset accuracy alone. A staged feature-leakage ablation improves cross-dataset transfer non-monotonically, indicating the leakage signal is distributed across the feature representation rather than confined to a few columns, and cross-dataset transfer between our two networks is strongly directional. These results argue for evaluating NIDS models along multiple independent robustness axes, and with more than one metric per axis.

Related papers

• Concept drift mitigation through community and spectral graph analysis for the detection of cyberattacks in network traffic [1.3999481573773072]
  We introduce t-robustness, a stability score defined for each feature independently of any detection model.<n>It combines the step-by-step distance between successive statistical states of a feature, and its cumulative divergence from its initial state, so that a slow monotonic drift cannot pass for stability.<n>The evaluation is performed on the UGR16 dataset, across three learning scenarios and a control scenario, as well as without model update.
  arXiv  Detail & Related papers  (2026-09-08T20:51:00Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Cross-Domain Generalization Failure in Lightweight Intrusion Detection Models for IIoT Networks [0.0]
  Lightweight machine learning models are increasingly proposed for intrusion detection in Industrial Internet of Things (IIoT) networks.<n>Most reported results evaluate these models only within their training network, leaving behavior on unseen networks unverified.<n>This study trains four lightweight architectures on one IIoT dataset and evaluates them, without retraining, on two structurally distinct IIoT datasets.
  arXiv  Detail & Related papers  (2026-07-01T07:43:26Z)
• Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement [66.67463557274358]
  Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes.<n>We introduce Confidence-guided Concept Mining (CBCM) to identify reliable spurious attributes without attribute annotations.<n>We propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches.
  arXiv  Detail & Related papers  (2026-06-23T05:28:47Z)
• Quantum Machine Learning for Cyber-Physical Anomaly Detection in Unmanned Aerial Vehicles: A Leakage-Free Evaluation with Proxy-Audited Feature Sets [0.0]
  Unmanned aerial vehicles (UAVs) are cyber-physical systems whose attack surface spans networked avionics and on-board sensor fusion.<n>We present a leakage-free evaluation of quantum machine learning for UAV anomaly detection on the multi-sensor TLM:UAV benchmark.
  arXiv  Detail & Related papers  (2026-05-19T01:05:16Z)
• UniAlign: A Model-Agnostic Framework for Robust Network Traffic Classification under Distribution Shifts [8.146941441861614]
  UniAlign is a model-agnostic framework that improves the robustness of deep learning-based NTC models under distribution shifts.<n>It can be seamlessly integrated into existing supervised NTC models without requiring specific feature modalities or introducing non-constant additional training costs.
  arXiv  Detail & Related papers  (2026-05-17T18:02:17Z)
• Cross-modal Proxy Evolving for OOD Detection with Vision-Language Models [59.242742594156546]
  CoEvo is a test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies.<n>CoEvo achieves state-of-the-art performance, improving AUROC by 1.33% and reducing FPR95 by 45.98% on ImageNet-1K compared to strong negative-label baselines.
  arXiv  Detail & Related papers  (2026-01-13T12:08:26Z)
• Diffusion Language Models are Super Data Learners [61.721441061210896]
  When unique data is limited, diffusion language models (DLMs) consistently surpass autoregressive (AR) models by training for more epochs.<n>We attribute the gains to three compounding factors: (1) any-order modeling, (2) super-dense compute from iterative bidirectional denoising, and (3) built-in Monte Carlo augmentation.
  arXiv  Detail & Related papers  (2025-11-05T08:17:42Z)
• Robust Fine-tuning of Zero-shot Models via Variance Reduction [56.360865951192324]
  When fine-tuning zero-shot models, our desideratum is for the fine-tuned model to excel in both in-distribution (ID) and out-of-distribution (OOD) We propose a sample-wise ensembling technique that can simultaneously attain the best ID and OOD accuracy without the trade-offs.
  arXiv  Detail & Related papers  (2024-11-11T13:13:39Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)
• False: False Negative Samples Aware Contrastive Learning for Semantic Segmentation of High-Resolution Remote Sensing Image [11.356381535900901]
  We propose a False negAtive sampLes aware contraStive lEarning model (FALSE) for the semantic segmentation of high-resolution RSIs. Since the SSCL pretraining is unsupervised, the lack of definable criteria for false negative sample (FNS) leads to theoretical undecidability. We achieve coarse determination of FNS by the FNS self-determination strategy and achieve calibration of FNS by the FNS confidence calibration (FNCC) loss function.
  arXiv  Detail & Related papers  (2022-11-15T06:24:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.