Binary Anomaly Detection in Streaming IoT Traffic under Concept Drift

• URL: http://arxiv.org/abs/2510.27304v1
• Date: Fri, 31 Oct 2025 09:21:28 GMT
• Title: Binary Anomaly Detection in Streaming IoT Traffic under Concept Drift
• Authors: Rodrigo Matos Carnier, Laura Lahesoo, Kensuke Fukuda,
• Abstract summary: Traditional batch learning models face challenges such as high maintenance and poor robustness to rapid anomaly changes.<n> streaming learning integrates online and incremental learning, enabling seamless updates and concept drift detection to improve adaptability.<n>This study investigates anomaly detection in streaming IoT traffic as binary classification, comparing batch and streaming learning approaches.
• Score: 2.650860836597657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the growing volume of Internet of Things (IoT) network traffic, machine learning (ML)-based anomaly detection is more relevant than ever. Traditional batch learning models face challenges such as high maintenance and poor adaptability to rapid anomaly changes, known as concept drift. In contrast, streaming learning integrates online and incremental learning, enabling seamless updates and concept drift detection to improve robustness. This study investigates anomaly detection in streaming IoT traffic as binary classification, comparing batch and streaming learning approaches while assessing the limitations of current IoT traffic datasets. We simulated heterogeneous network data streams by carefully mixing existing datasets and streaming the samples one by one. Our results highlight the failure of batch models to handle concept drift, but also reveal persisting limitations of current datasets to expose model limitations due to low traffic heterogeneity. We also investigated the competitiveness of tree-based ML algorithms, well-known in batch anomaly detection, and compared it to non-tree-based ones, confirming the advantages of the former. Adaptive Random Forest achieved F1-score of 0.990 $\pm$ 0.006 at one-third the computational cost of its batch counterpart. Hoeffding Adaptive Tree reached F1-score of 0.910 $\pm$ 0.007, reducing computational cost by four times, making it a viable choice for online applications despite a slight trade-off in stability.

Related papers

• Improving Real-Time Concept Drift Detection using a Hybrid Transformer-Autoencoder Framework [0.0]
  In applied machine learning, concept drift can significantly reduce model performance.<n>Our study proposes a hybrid framework consisting of Transformers and Autoencoders to model complex temporal dynamics.<n>Our results support that the Transformation-Autoencoder detected drift earlier and with more sensitivity than the autoencoders commonly used in the literature.
  arXiv  Detail & Related papers  (2025-08-09T19:39:33Z)
• ROSFD: Robust Online Streaming Fraud Detection with Resilience to Concept Drift in Data Streams [0.0]
  Continuous generation of streaming data necessitates timely fraud detection.<n>Traditional batch processing methods often struggle to capture the rapidly evolving patterns of fraudulent activities.<n>This paper highlights the critical importance of processing streaming data for effective fraud detection.
  arXiv  Detail & Related papers  (2025-04-14T13:50:23Z)
• Generative Active Adaptation for Drifting and Imbalanced Network Intrusion Detection [14.728689487990836]
  generative active adaptation framework minimizes labeling effort while enhancing model robustness.<n>We evaluate our end-to-end framework NetGuard on both simulated IDS data and a real-world ISP dataset.
  arXiv  Detail & Related papers  (2025-03-04T21:49:42Z)
• Towards Resource-Efficient Federated Learning in Industrial IoT for Multivariate Time Series Analysis [50.18156030818883]
  Anomaly and missing data constitute a thorny problem in industrial applications. Deep learning enabled anomaly detection has emerged as a critical direction. The data collected in edge devices contain user privacy.
  arXiv  Detail & Related papers  (2024-11-06T15:38:31Z)
• FLARE: Detection and Mitigation of Concept Drift for Federated Learning based IoT Deployments [2.7776688429637466]
  FLARE is a lightweight dual-scheduler FL framework that conditionally transfers training data and deploys models between edge and sensor endpoints. We show that FLARE can significantly reduce the amount of data exchanged between edge and sensor nodes compared to fixed-interval scheduling methods. It can successfully detect concept drift reactively with at least a 16x reduction in latency.
  arXiv  Detail & Related papers  (2023-05-15T10:09:07Z)
• Towards better traffic volume estimation: Jointly addressing the underdetermination and nonequilibrium problems with correlation-adaptive GNNs [47.18837782862979]
  This paper studies two key problems with regard to traffic volume estimation: (1) underdetermined traffic flows caused by undetected movements, and (2) non-equilibrium traffic flows arise from congestion propagation. We demonstrate a graph-based deep learning method that can offer a data-driven, model-free and correlation adaptive approach to tackle the above issues.
  arXiv  Detail & Related papers  (2023-03-10T02:22:33Z)
• Real-time Object Detection for Streaming Perception [84.2559631820007]
  Streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception. We build a simple and effective framework for streaming perception. Our method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline.
  arXiv  Detail & Related papers  (2022-03-23T11:33:27Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Incorporating Kinematic Wave Theory into a Deep Learning Method for High-Resolution Traffic Speed Estimation [3.0969191504482243]
  We propose a kinematic wave based Deep Convolutional Neural Network (Deep CNN) to estimate high resolution traffic speed dynamics from sparse probe vehicle trajectories. We introduce two key approaches that allow us to incorporate kinematic wave theory principles to improve the robustness of existing learning-based estimation methods.
  arXiv  Detail & Related papers  (2021-02-04T21:51:25Z)
• Prediction of Traffic Flow via Connected Vehicles [77.11902188162458]
  We propose a Short-term Traffic flow Prediction framework so that transportation authorities take early actions to control flow and prevent congestion. We anticipate flow at future time frames on a target road segment based on historical flow data and innovative features such as real time feeds and trajectory data provided by Connected Vehicles (CV) technology. We show how this novel approach allows advanced modelling by integrating into the forecasting of flow, the impact of various events that CV realistically encountered on segments along their trajectory.
  arXiv  Detail & Related papers  (2020-07-10T16:00:44Z)
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
  We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass. We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
  arXiv  Detail & Related papers  (2020-03-04T12:27:36Z)
• Adaptive Anomaly Detection for IoT Data in Hierarchical Edge Computing [71.86955275376604]
  We propose an adaptive anomaly detection approach for hierarchical edge computing (HEC) systems to solve this problem. We design an adaptive scheme to select one of the models based on the contextual information extracted from input data, to perform anomaly detection. We evaluate our proposed approach using a real IoT dataset, and demonstrate that it reduces detection delay by 84% while maintaining almost the same accuracy as compared to offloading detection tasks to the cloud.
  arXiv  Detail & Related papers  (2020-01-10T05:29:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.