FedBiF: Communication-Efficient Federated Learning via Bits Freezing
- URL: http://arxiv.org/abs/2509.10161v1
- Date: Fri, 12 Sep 2025 11:41:06 GMT
- Title: FedBiF: Communication-Efficient Federated Learning via Bits Freezing
- Authors: Shiwei Li, Qunwei Li, Haozhao Wang, Ruixuan Li, Jianbin Lin, Wenliang Zhong,
- Abstract summary: Federated Bit Freezing (FedBiF) is a novel FL framework that directly learns quantized model parameters during local training.<n>FedBiF achieves superior communication compression and also promotes sparsity in the resulting models.
- Score: 30.27100790578667
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative model training without sharing local data. Despite its advantages, FL suffers from substantial communication overhead, which can affect training efficiency. Recent efforts have mitigated this issue by quantizing model updates to reduce communication costs. However, most existing methods apply quantization only after local training, introducing quantization errors into the trained parameters and potentially degrading model accuracy. In this paper, we propose Federated Bit Freezing (FedBiF), a novel FL framework that directly learns quantized model parameters during local training. In each communication round, the server first quantizes the model parameters and transmits them to the clients. FedBiF then allows each client to update only a single bit of the multi-bit parameter representation, freezing the remaining bits. This bit-by-bit update strategy reduces each parameter update to one bit while maintaining high precision in parameter representation. Extensive experiments are conducted on five widely used datasets under both IID and Non-IID settings. The results demonstrate that FedBiF not only achieves superior communication compression but also promotes sparsity in the resulting models. Notably, FedBiF attains accuracy comparable to FedAvg, even when using only 1 bit-per-parameter (bpp) for uplink and 3 bpp for downlink communication. The code is available at https://github.com/Leopold1423/fedbif-tpds25.
Related papers
- Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models [54.02863371927658]
Large Language Models (LLMs) have become indispensable in numerous real-world applications.<n>Ferret is the first first-order method with shared randomness to enable scalable full- parameter tuning of LLMs.<n>Ferret achieves high computational efficiency, reduced communication overhead, and fast convergence.
arXiv Detail & Related papers (2024-09-10T07:28:13Z) - SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead [75.87007729801304]
SpaFL: a communication-efficient FL framework is proposed to optimize sparse model structures with low computational overhead.<n>To optimize the pruning process itself, only thresholds are communicated between a server and clients instead of parameters.<n>Global thresholds are used to update model parameters by extracting aggregated parameter importance.
arXiv Detail & Related papers (2024-06-01T13:10:35Z) - Fed-CVLC: Compressing Federated Learning Communications with
Variable-Length Codes [54.18186259484828]
In Federated Learning (FL) paradigm, a parameter server (PS) concurrently communicates with distributed participating clients for model collection, update aggregation, and model distribution over multiple rounds.
We show strong evidences that variable-length is beneficial for compression in FL.
We present Fed-CVLC (Federated Learning Compression with Variable-Length Codes), which fine-tunes the code length in response to the dynamics of model updates.
arXiv Detail & Related papers (2024-02-06T07:25:21Z) - Parametric Feature Transfer: One-shot Federated Learning with Foundation
Models [14.97955440815159]
In one-shot federated learning, clients collaboratively train a global model in a single round of communication.
This paper introduces FedPFT, a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL.
arXiv Detail & Related papers (2024-02-02T19:34:46Z) - ResFed: Communication Efficient Federated Learning by Transmitting Deep
Compressed Residuals [24.13593410107805]
Federated learning enables cooperative training among massively distributed clients by sharing their learned local model parameters.
We introduce a residual-based federated learning framework (ResFed), where residuals rather than model parameters are transmitted in communication networks for training.
By employing a common prediction rule, both locally and globally updated models are always fully recoverable in clients and the server.
arXiv Detail & Related papers (2022-12-11T20:34:52Z) - FiT: Parameter Efficient Few-shot Transfer Learning for Personalized and
Federated Image Classification [47.24770508263431]
We develop FiLM Transfer (FiT) which fulfills requirements in the image classification setting.
FiT uses an automatically configured Naive Bayes classifier on top of a fixed backbone that has been pretrained on large image datasets.
We show that FiT achieves better classification accuracy than the state-of-the-art Big Transfer (BiT) algorithm at low-shot and on the challenging VTAB-1k benchmark.
arXiv Detail & Related papers (2022-06-17T10:17:20Z) - Acceleration of Federated Learning with Alleviated Forgetting in Local
Training [61.231021417674235]
Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy.
We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage.
Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
arXiv Detail & Related papers (2022-03-05T02:31:32Z) - Fast Server Learning Rate Tuning for Coded Federated Dropout [3.9653673778225946]
Federated Dropout (FD) is a technique that improves the communication efficiency of a FL session.
We leverage coding theory to enhance FD by allowing a different sub-model to be used at each client.
For the EMNIST dataset, our mechanism achieves 99.6 % of the final accuracy of the no dropout case.
arXiv Detail & Related papers (2022-01-26T16:19:04Z) - Communication-Efficient Federated Learning for Neural Machine
Translation [1.5362025549031046]
Training neural machine translation (NMT) models in federated learning (FL) settings could be inefficient both computationally and communication-wise.
In this paper, we explore how to efficiently build NMT models in an FL setup by proposing a novel solution.
In order to reduce the communication overhead, out of all neural layers we only exchange what we term "Controller" layers.
arXiv Detail & Related papers (2021-12-12T03:16:03Z) - Over-the-Air Federated Learning from Heterogeneous Data [107.05618009955094]
Federated learning (FL) is a framework for distributed learning of centralized models.
We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local gradient descent (SGD) FL algorithm.
We numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL.
arXiv Detail & Related papers (2020-09-27T08:28:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.