AutoFlow: An Autoencoder-based Approach for IP Flow Record Compression with Minimal Impact on Traffic Classification

• URL: http://arxiv.org/abs/2410.00030v1
• Date: Tue, 17 Sep 2024 11:24:22 GMT
• Title: AutoFlow: An Autoencoder-based Approach for IP Flow Record Compression with Minimal Impact on Traffic Classification
• Authors: Adrian Pekar,
• Abstract summary: This paper presents a novel approach to compressing IP flow records using deep learning techniques, specifically autoencoders. We demonstrate the effectiveness of our approach through extensive experiments on a large-scale, real-world network traffic dataset. The implications of this work extend to more efficient network monitoring, real-time analysis in resource-constrained environments, and scalable network management solutions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a novel approach to compressing IP flow records using deep learning techniques, specifically autoencoders. Our method aims to significantly reduce data volume while maintaining the utility of the compressed data for downstream analysis tasks. We demonstrate the effectiveness of our approach through extensive experiments on a large-scale, real-world network traffic dataset. The proposed autoencoder-based compression achieves a 3.28x reduction in data size while preserving 99.20% accuracy in a multi-class traffic classification task, compared to 99.77% accuracy with uncompressed data. This marginal decrease in performance is offset by substantial gains in storage efficiency and potential improvements in processing speed. Our method shows particular promise in distinguishing between various modern application protocols, including encrypted traffic from popular services. The implications of this work extend to more efficient network monitoring, real-time analysis in resource-constrained environments, and scalable network management solutions.

Related papers

• Channel-Aware Throughput Maximization for Cooperative Data Fusion in CAV [17.703608985129026]
  Connected and autonomous vehicles (CAVs) have garnered significant attention due to their extended perception range and enhanced sensing coverage. To address challenges such as blind spots and obstructions, CAVs employ vehicle-to-vehicle communications to aggregate data from surrounding vehicles. We propose a channel-aware throughput approach to facilitate CAV data fusion, leveraging a self-supervised autoencoder for adaptive data compression.
  arXiv  Detail & Related papers  (2024-10-06T00:43:46Z)
• Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning [29.727339562140653]
  Current data compression methods, such as sparsification in Federated Averaging (FedAvg), effectively enhance the communication efficiency of Federated Learning (FL) These methods encounter challenges such as the straggler problem and diminished model performance due to heterogeneous bandwidth and non-IID data. We introduce a bandwidth-aware compression framework for FL, aimed at improving communication efficiency while mitigating the problems associated with non-IID data.
  arXiv  Detail & Related papers  (2024-08-27T02:28:27Z)
• Enabling robust sensor network design with data processing and optimization making use of local beehive image and video files [0.0]
  We of er a revolutionary paradigm that uses cutting-edge edge computing techniques to optimize data transmission and storage. Our approach encompasses data compression for images and videos, coupled with a data aggregation technique for numerical data. A key aspect of our approach is its ability to operate in resource-constrained environments.
  arXiv  Detail & Related papers  (2024-02-26T15:27:47Z)
• Accelerating Distributed Deep Learning using Lossless Homomorphic Compression [17.654138014999326]
  We introduce a novel compression algorithm that effectively merges worker-level compression with in-network aggregation. We show up to a 6.33$times$ improvement in aggregation throughput and a 3.74$times$ increase in per-iteration training speed.
  arXiv  Detail & Related papers  (2024-02-12T09:57:47Z)
• Task-aware Distributed Source Coding under Dynamic Bandwidth [24.498190179263837]
  We propose a distributed compression framework composed of independent encoders and a joint decoder, which we call neural distributed principal component analysis (NDPCA) NDPCA flexibly compresses data from multiple sources to any available bandwidth with a single model, reducing computing and storage overhead. Experiments show that NDPCA improves the success rate of multi-view robotic arm manipulation by 9% and the accuracy of object detection tasks on satellite imagery by 14%.
  arXiv  Detail & Related papers  (2023-05-24T19:20:59Z)
• Optimal transfer protocol by incremental layer defrosting [66.76153955485584]
  Transfer learning is a powerful tool enabling model training with limited amounts of data. The simplest transfer learning protocol is based on freezing" the feature-extractor layers of a network pre-trained on a data-rich source task. We show that this protocol is often sub-optimal and the largest performance gain may be achieved when smaller portions of the pre-trained network are kept frozen.
  arXiv  Detail & Related papers  (2023-03-02T17:32:11Z)
• Attention-based Feature Compression for CNN Inference Offloading in Edge Computing [93.67044879636093]
  This paper studies the computational offloading of CNN inference in device-edge co-inference systems. We propose a novel autoencoder-based CNN architecture (AECNN) for effective feature extraction at end-device. Experiments show that AECNN can compress the intermediate data by more than 256x with only about 4% accuracy loss.
  arXiv  Detail & Related papers  (2022-11-24T18:10:01Z)
• Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach [76.45949280328838]
  We propose a Laplacian enhanced low-rank tensor (LETC) framework featuring both lowrankness and multi-temporal correlations for large-scale traffic speed kriging. We then design an efficient solution algorithm via several effective numeric techniques to scale up the proposed model to network-wide kriging.
  arXiv  Detail & Related papers  (2022-10-21T07:25:57Z)
• Unrolled Compressed Blind-Deconvolution [77.88847247301682]
  sparse multichannel blind deconvolution (S-MBD) arises frequently in many engineering applications such as radar/sonar/ultrasound imaging. We propose a compression method that enables blind recovery from much fewer measurements with respect to the full received signal in time.
  arXiv  Detail & Related papers  (2022-09-28T15:16:58Z)
• ALF: Autoencoder-based Low-rank Filter-sharing for Efficient Convolutional Neural Networks [63.91384986073851]
  We propose the autoencoder-based low-rank filter-sharing technique technique (ALF) ALF shows a reduction of 70% in network parameters, 61% in operations and 41% in execution time, with minimal loss in accuracy.
  arXiv  Detail & Related papers  (2020-07-27T09:01:22Z)
• OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression [77.8842824702423]
  We present a novel deep compression algorithm to reduce the memory footprint of LiDAR point clouds. Our method exploits the sparsity and structural redundancy between points to reduce the memory footprint. Our algorithm can be used to reduce the onboard and offboard storage of LiDAR points for applications such as self-driving cars.
  arXiv  Detail & Related papers  (2020-05-14T17:48:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.