Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning

• URL: http://arxiv.org/abs/2511.05593v1
• Date: Wed, 05 Nov 2025 13:11:30 GMT
• Title: Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning
• Authors: Arnaud Descours, Léonard Deroose, Jan Ramon,
• Abstract summary: Federated Learning (FL) enables decentralized model training across multiple clients while preserving data privacy.<n>We introduce two algorithms: ProjFL, designed for unbiased compressors, and ProjFL+EF, for biased compressors through an Error Feedback mechanism.
• Score: 0.8220217498103312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) enables decentralized model training across multiple clients while optionally preserving data privacy. However, communication efficiency remains a critical bottleneck, particularly for large-scale models. In this work, we introduce two complementary algorithms: ProjFL, designed for unbiased compressors, and ProjFL+EF, tailored for biased compressors through an Error Feedback mechanism. Both methods rely on projecting local gradients onto a shared client-server subspace spanned by historical descent directions, enabling efficient information exchange with minimal communication overhead. We establish convergence guarantees for both algorithms under strongly convex, convex, and non-convex settings. Empirical evaluations on standard FL classification benchmarks with deep neural networks show that ProjFL and ProjFL+EF achieve accuracy comparable to existing baselines while substantially reducing communication costs.

Related papers

• Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching [19.959825317183924]
  Federated Learning (FL) enables collaborative training across decentralized data.<n>We propose pFed1BS, a novel personalized FL framework that achieves extreme communication compression through one-bit random sketching.<n>In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus.
  arXiv  Detail & Related papers  (2025-11-17T08:55:22Z)
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks [51.43780477302533]
  Contribution-Oriented PFL (CO-PFL) is a novel algorithm that dynamically estimates each client's contribution for global aggregation.<n>CO-PFL consistently surpasses state-of-the-art methods in robustness in personalization accuracy, robustness, scalability and convergence stability.
  arXiv  Detail & Related papers  (2025-10-23T05:10:06Z)
• Over-the-Air Fair Federated Learning via Multi-Objective Optimization [52.295563400314094]
  We propose an over-the-air fair federated learning algorithm (OTA-FFL) to train fair FL models.<n>Experiments demonstrate the superiority of OTA-FFL in achieving fairness and robust performance.
  arXiv  Detail & Related papers  (2025-01-06T21:16:51Z)
• Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration [66.43954501171292]
  We introduce Catalyst Acceleration and propose an acceleration Decentralized Federated Learning algorithm called DFedCata. DFedCata consists of two main components: the Moreau envelope function, which addresses parameter inconsistencies, and Nesterov's extrapolation step, which accelerates the aggregation phase. Empirically, we demonstrate the advantages of the proposed algorithm in both convergence speed and generalization performance on CIFAR10/100 with various non-iid data distributions.
  arXiv  Detail & Related papers  (2024-10-09T06:17:16Z)
• Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets [25.010661914466354]
  In a real federated learning (FL) system, communication overhead for passing model parameters between the clients and the parameter (PS) is often a bottleneck. We propose sequential FL (SFL) HFL for the first time, which removes the central PS and enables the model to be completed only through passing data between two adjacent ESs for each server.
  arXiv  Detail & Related papers  (2024-08-19T07:43:35Z)
• SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead [75.87007729801304]
  SpaFL: a communication-efficient FL framework is proposed to optimize sparse model structures with low computational overhead.<n>To optimize the pruning process itself, only thresholds are communicated between a server and clients instead of parameters.<n>Global thresholds are used to update model parameters by extracting aggregated parameter importance.
  arXiv  Detail & Related papers  (2024-06-01T13:10:35Z)
• Decentralized Directed Collaboration for Personalized Federated Learning [39.29794569421094]
  We concentrate on the Decentralized Personalized Learning (DPFL) that performs distributed training model computation. We propose a directed collaboration framework by incorporating textbfDecentralized textbfFederated textbfPartial textbfGradient textbfPedGP.
  arXiv  Detail & Related papers  (2024-05-28T06:52:19Z)
• CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models [21.919883617413358]
  This study introduces an innovative approach to compress gradients to improve communication efficiency during Large-Language Models (LLMs) We also present a series of experimental analyses focusing on the signal-to-noise ratio, compression rate, and robustness within this privacy-centric framework.
  arXiv  Detail & Related papers  (2024-05-22T15:32:38Z)
• FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models [52.13056951033747]
  Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server.<n>A critical bottleneck in FL is the communication cost.<n>Our work is inspired by the innovative Scaffnew algorithm, which has advanced the reduction of communication complexity in FL.<n>We introduce FedComLoc, integrating practical and effective compression into Scaffnew to further enhance communication efficiency.
  arXiv  Detail & Related papers  (2024-03-14T22:29:59Z)
• Federated Learning with Reduced Information Leakage and Computation [17.069452700698047]
  Federated learning (FL) is a distributed learning paradigm that allows multiple decentralized clients to collaboratively learn a common model without sharing local data. This paper introduces Upcycled-FL, a strategy that applies first-order approximation at every even round of model update. Under this strategy, half of the FL updates incur no information leakage and require much less computational and transmission costs.
  arXiv  Detail & Related papers  (2023-10-10T06:22:06Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Vertical Federated Learning over Cloud-RAN: Convergence Analysis and System Optimization [82.12796238714589]
  We propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation. We characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. We establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed.
  arXiv  Detail & Related papers  (2023-05-04T09:26:03Z)
• Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization [21.81192774458227]
  One of the major bottlenecks is the large communication cost between the central server and the local workers. Our proposed distributed learning framework features an effective gradient gradient compression strategy.
  arXiv  Detail & Related papers  (2021-11-01T04:54:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.