FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization

• URL: http://arxiv.org/abs/2506.23516v3
• Date: Tue, 22 Jul 2025 03:52:09 GMT
• Title: FedWSQ: Efficient Federated Learning with Weight Standardization and Distribution-Aware Non-Uniform Quantization
• Abstract summary: Federated learning (FL) often suffers from performance degradation due to key challenges such as data heterogeneity and communication constraints.<n>We present a novel FL framework called FedWSQ, which integrates weight standardization (WS) and the proposed distribution-aware non-uniform quantization (DANUQ)
• Score: 4.487640233047808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning (FL) often suffers from performance degradation due to key challenges such as data heterogeneity and communication constraints. To address these limitations, we present a novel FL framework called FedWSQ, which integrates weight standardization (WS) and the proposed distribution-aware non-uniform quantization (DANUQ). WS enhances FL performance by filtering out biased components in local updates during training, thereby improving the robustness of the model against data heterogeneity and unstable client participation. In addition, DANUQ minimizes quantization errors by leveraging the statistical properties of local model updates. As a result, FedWSQ significantly reduces communication overhead while maintaining superior model accuracy. Extensive experiments on FL benchmark datasets demonstrate that FedWSQ consistently outperforms existing FL methods across various challenging FL settings, including extreme data heterogeneity and ultra-low-bit communication scenarios.

Related papers

• FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Data [0.2840208039211775]
  Federated learning (FL) enables a set of distributed clients to jointly train machine learning models.<n>We propose FedPBS, an FL algorithm that couples complementary ideas from FedBS and FedProx to address these challenges.
  arXiv  Detail & Related papers  (2026-03-14T11:44:27Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarios [14.18492489954482]
  Federated Learning (FL) enables decentralized model training while preserving data privacy.<n>Despite its benefits, FL faces challenges with non-identically distributed (non-IID) data.<n>We propose FedWCM, a method that dynamically adjusts momentum using global and per-round data.
  arXiv  Detail & Related papers  (2025-07-20T14:24:57Z)
• Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning [3.120955853908236]
  Federated Learning (FL) enables collaborative model training across decentralized clients without sharing raw data.<n>We introduce ShiftEx, a shift-aware mixture of experts framework that creates and trains specialized global models in response to detected distribution shifts.<n>We demonstrate 5.5-12.9 percentage point accuracy improvements and 22-95 % faster adaptation compared to state-of-the-art FL baselines across diverse shift scenarios.
  arXiv  Detail & Related papers  (2025-06-23T15:59:21Z)
• FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments [4.358456799125694]
  Pervasive computing environments (e.g., for Human Activity Recognition, HAR) are characterized by resource-constrained end devices, streaming sensor data and intermittent client participation.<n>We propose FlexFed, a novel FL approach that prioritizes data retention for efficient memory use and dynamically adjusts offline training frequency.<n>We also develop a realistic HAR-based evaluation framework that simulates streaming data, dynamic distributions, imbalances and varying availability.
  arXiv  Detail & Related papers  (2025-05-19T14:23:37Z)
• Client Contribution Normalization for Enhanced Federated Learning [4.726250115737579]
  Mobile devices, including smartphones and laptops, generate decentralized and heterogeneous data. Federated Learning (FL) offers a promising alternative by enabling collaborative training of a global model across decentralized devices without data sharing. This paper focuses on data-dependent heterogeneity in FL and proposes a novel approach leveraging mean latent representations extracted from locally trained models.
  arXiv  Detail & Related papers  (2024-11-10T04:03:09Z)
• Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning? [50.03434441234569]
  Federated Learning (FL) has gained significant popularity due to its effectiveness in training machine learning models across diverse sites without requiring direct data sharing. While various algorithms have shown that FL with local updates is a communication-efficient distributed learning framework, the generalization performance of FL with local updates has received comparatively less attention.
  arXiv  Detail & Related papers  (2024-09-05T19:00:18Z)
• FedHPL: Efficient Heterogeneous Federated Learning with Prompt Tuning and Logit Distillation [32.305134875959226]
  Federated learning (FL) is a privacy-preserving paradigm that enables distributed clients to collaboratively train models with a central server. We propose FedHPL, a parameter-efficient unified $textbfFed$erated learning framework for $textbfH$eterogeneous settings. We show that our framework outperforms state-of-the-art FL approaches, with less overhead and training rounds.
  arXiv  Detail & Related papers  (2024-05-27T15:25:32Z)
• FedShift: Robust Federated Learning Aggregation Scheme in Resource Constrained Environment via Weight Shifting [5.680416078423551]
  Federated Learning (FL) commonly relies on a central server to coordinate training across distributed clients.<n>Clients may employ different quantization levels based on their hardware or network constraints, necessitating a mixed-precision aggregation process at the server.<n>We propose FedShift, a novel aggregation methodology designed to mitigate performance degradation in FL scenarios with mixed quantization levels.
  arXiv  Detail & Related papers  (2024-02-02T00:03:51Z)
• Adaptive Model Pruning and Personalization for Federated Learning over Wireless Networks [72.59891661768177]
  Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device.
  arXiv  Detail & Related papers  (2023-09-04T21:10:45Z)
• Scheduling and Aggregation Design for Asynchronous Federated Learning over Wireless Networks [56.91063444859008]
  Federated Learning (FL) is a collaborative machine learning framework that combines on-device training and server-based aggregation. We propose an asynchronous FL design with periodic aggregation to tackle the straggler issue in FL systems. We show that an age-aware'' aggregation weighting design can significantly improve the learning performance in an asynchronous FL setting.
  arXiv  Detail & Related papers  (2022-12-14T17:33:01Z)
• Performance Optimization for Variable Bitwidth Federated Learning in Wireless Networks [103.22651843174471]
  This paper considers improving wireless communication and computation efficiency in federated learning (FL) via model quantization. In the proposed bitwidth FL scheme, edge devices train and transmit quantized versions of their local FL model parameters to a coordinating server, which aggregates them into a quantized global model and synchronizes the devices. We show that the FL training process can be described as a Markov decision process and propose a model-based reinforcement learning (RL) method to optimize action selection over iterations.
  arXiv  Detail & Related papers  (2022-09-21T08:52:51Z)
• ClusterQ: Semantic Feature Distribution Alignment for Data-Free Quantization [111.12063632743013]
  We propose a new and effective data-free quantization method termed ClusterQ. To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics. We also incorporate the intra-class variance to solve class-wise mode collapse.
  arXiv  Detail & Related papers  (2022-04-30T06:58:56Z)
• Feature Quantization Improves GAN Training [126.02828112121874]
  Feature Quantization (FQ) for the discriminator embeds both true and fake data samples into a shared discrete space. Our method can be easily plugged into existing GAN models, with little computational overhead in training.
  arXiv  Detail & Related papers  (2020-04-05T04:06:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.