Efficient Federated Learning Tiny Language Models for Mobile Network Feature Prediction

• URL: http://arxiv.org/abs/2504.01947v1
• Date: Wed, 02 Apr 2025 17:54:06 GMT
• Title: Efficient Federated Learning Tiny Language Models for Mobile Network Feature Prediction
• Authors: Daniel Becking, Ingo Friese, Karsten Müller, Thomas Buchholz, Mandy Galkow-Schneider, Wojciech Samek, Detlev Marpe,
• Abstract summary: In telecommunications, Autonomous Networks (ANs) automatically adjust configurations based on specific requirements (e.g. bandwidth, available resources)<n>Here, Federated Learning (FL) allows multiple AN cells - each equipped with Neural Networks (NNs) - to collaboratively train models while preserving data privacy.<n>We investigate NNCodec, a implementation of the ISO/IEC Neural Network Coding (NNC) standard, within a novel FL framework that integrates tiny language models (TLMs)<n>Our experimental results on the Berlin V2X dataset demonstrate that NNCodec achieves transparent compression while reducing communication overhead to below 1%.
• Score: 13.32608465848856
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In telecommunications, Autonomous Networks (ANs) automatically adjust configurations based on specific requirements (e.g., bandwidth) and available resources. These networks rely on continuous monitoring and intelligent mechanisms for self-optimization, self-repair, and self-protection, nowadays enhanced by Neural Networks (NNs) to enable predictive modeling and pattern recognition. Here, Federated Learning (FL) allows multiple AN cells - each equipped with NNs - to collaboratively train models while preserving data privacy. However, FL requires frequent transmission of large neural data and thus an efficient, standardized compression strategy for reliable communication. To address this, we investigate NNCodec, a Fraunhofer implementation of the ISO/IEC Neural Network Coding (NNC) standard, within a novel FL framework that integrates tiny language models (TLMs) for various mobile network feature prediction (e.g., ping, SNR or band frequency). Our experimental results on the Berlin V2X dataset demonstrate that NNCodec achieves transparent compression (i.e., negligible performance loss) while reducing communication overhead to below 1%, showing the effectiveness of combining NNC with FL in collaboratively learned autonomous mobile networks.

Related papers

• Learning Load Balancing with GNN in MPTCP-Enabled Heterogeneous Networks [13.178956651532213]
  We propose a graph neural network (GNN)-based model to tackle the LB problem for MP TCP-enabled HetNets. Compared to the conventional deep neural network (DNN), the proposed GNN-based model exhibits two key strengths.
  arXiv  Detail & Related papers  (2024-10-22T15:49:53Z)
• The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning [6.9569682335746235]
  Spiking Neural Networks (SNNs) have recently gained significant interest in on-chip learning in embedded devices. In this paper, we explore the inherent robustness of SNNs under noisy communication in Federated Learning. We propose a novel Federated Learning with TopK Sparsification algorithm to reduce the bandwidth usage for FL training.
  arXiv  Detail & Related papers  (2024-09-19T13:37:18Z)
• Hyperdimensional Computing Empowered Federated Foundation Model over Wireless Networks for Metaverse [56.384390765357004]
  We propose an integrated federated split learning and hyperdimensional computing framework for emerging foundation models. This novel approach reduces communication costs, computation load, and privacy risks, making it suitable for resource-constrained edge devices in the Metaverse.
  arXiv  Detail & Related papers  (2024-08-26T17:03:14Z)
• Improving the Real-Data Driven Network Evaluation Model for Digital Twin Networks [0.2499907423888049]
  Digital Twin Networks (DTN) technology is expected to become the foundation technology for autonomous networks. DTN has the advantage of being able to operate and system networks based on real-time collected data in a closed-loop system. Various AI research and standardization work is ongoing to optimize the use of DTN.
  arXiv  Detail & Related papers  (2024-05-14T09:55:03Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• FedDCT: A Dynamic Cross-Tier Federated Learning Framework in Wireless Networks [5.914766366715661]
  Federated Learning (FL) trains a global model across devices without exposing local data. resource heterogeneity and inevitable stragglers in wireless networks severely impact the efficiency and accuracy of FL training. We propose a novel Dynamic Cross-Tier Federated Learning framework (FedDCT)
  arXiv  Detail & Related papers  (2023-07-10T08:54:07Z)
• SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networks [56.68149211499535]
  Federated learning (FL) is a key enabler for efficient communication and computing leveraging devices' distributed computing capabilities. This paper proposes a novel learning framework by integrating FL and width-adjustable slimmable neural networks (SNNs) We propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models.
  arXiv  Detail & Related papers  (2022-03-26T15:06:13Z)
• Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks [52.93232352968347]
  This paper aims to integrate two synergetic technologies, federated learning (FL) and width-adjustable slimmable neural network (SNN) FL preserves data privacy by exchanging the locally trained models of mobile devices. SNNs are however non-trivial, particularly under wireless connections with time-varying channel conditions. We propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models.
  arXiv  Detail & Related papers  (2021-12-05T11:17:17Z)
• Communication-Efficient Federated Learning with Binary Neural Networks [15.614120327271557]
  Federated learning (FL) is a privacy-preserving machine learning setting. FL involves a frequent exchange of the parameters between all the clients and the server that coordinates the training. In this paper, we consider training the binary neural networks (BNN) in the FL setting instead of the typical real-valued neural networks.
  arXiv  Detail & Related papers  (2021-10-05T15:59:49Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Flexible Transmitter Network [84.90891046882213]
  Current neural networks are mostly built upon the MP model, which usually formulates the neuron as executing an activation function on the real-valued weighted aggregation of signals received from other neurons. We propose the Flexible Transmitter (FT) model, a novel bio-plausible neuron model with flexible synaptic plasticity. We present the Flexible Transmitter Network (FTNet), which is built on the most common fully-connected feed-forward architecture.
  arXiv  Detail & Related papers  (2020-04-08T06:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.