Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data

• URL: http://arxiv.org/abs/2603.07654v1
• Date: Sun, 08 Mar 2026 14:37:16 GMT
• Title: Compressed Proximal Federated Learning for Non-Convex Composite Optimization on Heterogeneous Data
• Abstract summary: FCO achieves communication efficiency and robustness when dealing with non-smoothizers.<n>Fed Composite Error FeedbackCEF is a novel algorithm for non-efficient quant convergence.<n>It mitigates the noise from aggressiveization and the noise from non-IID data.
• Score: 12.4809003470908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Composite Optimization (FCO) has emerged as a promising framework for training models with structural constraints (e.g., sparsity) in distributed edge networks. However, simultaneously achieving communication efficiency and convergence robustness remains a significant challenge, particularly when dealing with non-smooth regularizers, statistical heterogeneity, and the restrictions of biased compression. To address these issues, we propose FedCEF (Federated Composite Error Feedback), a novel algorithm tailored for non-convex FCO. FedCEF introduces a decoupled proximal update scheme that separates the proximal operator from communication, enabling clients to handle non-smooth terms locally while transmitting compressed information. To mitigate the noise from aggressive quantization and the bias from non-IID data, FedCEF integrates a rigorous error feedback mechanism with control variates. Furthermore, we design a communication-efficient pre-proximal downlink strategy that allows clients to exactly reconstruct global control variables without explicit transmission. We theoretically establish that FedCEF achieves sublinear convergence to a bounded residual error under general non-convexity, which is controllable via the step size and batch size. Extensive experiments on real datasets validate FedCEF maintains competitive model accuracy even under extreme compression ratios (e.g., 1%), significantly reducing the total communication volume compared to uncompressed baselines.

Related papers

• FedZMG: Efficient Client-Side Optimization in Federated Learning [0.19116784879310023]
  Federated Zero Mean Gradients (FedZMG) is a parameter-free, client-side optimization algorithm designed to tackle client-drift.<n>FedZMG projects local gradients onto a zero-mean hyperplane, effectively neutralizing the "intensity" or "bias" shifts inherent in heterogeneous data distributions.
  arXiv  Detail & Related papers  (2026-02-20T17:45:28Z)
• Fractional-Order Federated Learning [4.1751058176413105]
  Federated learning (FL) allows remote clients to train a global model collaboratively while protecting client privacy.<n>Despite its privacy-preserving benefits, FL has significant drawbacks, including slow convergence, high communication cost, and non-independent-and-identically-distributed (non-IID) data.
  arXiv  Detail & Related papers  (2026-02-17T06:25:23Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• FedSparQ: Adaptive Sparse Quantization with Error Feedback for Robust & Efficient Federated Learning [7.461859467262201]
  Federated Learning (FL) enables collaborative model training across decentralized clients.<n>FL suffers from significant communication overhead due to the frequent exchange of high-dimensional model updates over constrained networks.<n>We present FedSparQ, a lightweight compression framework that dynamically sparsifies the gradient of each client.
  arXiv  Detail & Related papers  (2025-11-05T12:38:08Z)
• Mitigating Catastrophic Forgetting with Adaptive Transformer Block Expansion in Federated Fine-Tuning [25.121545962121907]
  Federated fine-tuning (FedFT) of large language models (LLMs) has emerged as a promising solution for adapting models to distributed data environments.<n>We propose FedBE, a novel FedFT framework that integrates an adaptive transformer block expansion mechanism with a dynamic trainable-block allocation strategy.<n>We show that FedBE achieves 12-74% higher accuracy retention on general tasks after fine-tuning and a model convergence acceleration ratio of 1.9-3.1x without degrading the accuracy of downstream tasks.
  arXiv  Detail & Related papers  (2025-06-06T10:59:11Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• FedCanon: Non-Convex Composite Federated Learning with Efficient Proximal Operation on Heterogeneous Data [17.80715992954134]
  Composite learning offers a general framework for solving machine learning problems with additional regularization terms.<n>We propose FedCanon algorithm to solve possibly non-smooth regularization problems.
  arXiv  Detail & Related papers  (2025-04-16T09:28:26Z)
• Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks [55.467288506826755]
  Federated learning (FL) has been recognized as a viable solution for local-privacy-aware collaborative model training in wireless edge networks.<n>Most existing communication-efficient FL algorithms fail to reduce the significant inter-device variance.<n>We propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme.
  arXiv  Detail & Related papers  (2025-01-20T04:26:21Z)
• Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes [54.18186259484828]
  In Federated Learning (FL) paradigm, a parameter server (PS) concurrently communicates with distributed participating clients for model collection, update aggregation, and model distribution over multiple rounds. We show strong evidences that variable-length is beneficial for compression in FL. We present Fed-CVLC (Federated Learning Compression with Variable-Length Codes), which fine-tunes the code length in response to the dynamics of model updates.
  arXiv  Detail & Related papers  (2024-02-06T07:25:21Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation [95.85026305874824]
  We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
  arXiv  Detail & Related papers  (2022-12-14T13:57:01Z)
• Faster Non-Convex Federated Learning via Global and Local Momentum [57.52663209739171]
  textttFedGLOMO is the first (first-order) FLtexttFedGLOMO algorithm. Our algorithm is provably optimal even with communication between the clients and the server.
  arXiv  Detail & Related papers  (2020-12-07T21:05:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.