FedImp: Enhancing Federated Learning Convergence with Impurity-Based Weighting

• URL: http://arxiv.org/abs/2608.14654v1
• Date: Fri, 31 Jul 2026 05:46:03 GMT
• Title: FedImp: Enhancing Federated Learning Convergence with Impurity-Based Weighting
• Abstract summary: Federated Learning (FL) enables devices to train a global model while preserving local data privacy.<n>A major challenge in FL is the non-Independent and Identically Distributed (non-IID) nature of data across devices.<n>We propose Federated Impurity Weighting (FedImp), a novel algorithm that quantifies each device contribution based on the informational content of its local data.
• Score: 2.441001020367009
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) is a collaborative paradigm that enables multiple devices to train a global model while preserving local data privacy. A major challenge in FL is the non-Independent and Identically Distributed (non-IID) nature of data across devices, which hinders training efficiency and slows convergence. To tackle this, we propose Federated Impurity Weighting (FedImp), a novel algorithm that quantifies each device contribution based on the informational content of its local data. These contributions are normalized to compute distinct aggregation weights for the global model update. Extensive experiments on EMNIST and CIFAR-10 datasets show that FedImp significantly improves convergence speed, reducing communication rounds by up to 64.4%, 27.8%, and 66.7% on EMNIST, and 44.2%, 44%, and 25.6% on CIFAR-10 compared to FedAvg, FedProx, and FedAdp, respectively. Under highly imbalanced data distributions, FedImp outperforms all baselines and achieves the highest accuracy. Overall, FedImp offers an effective solution to enhance FL efficiency in non-IID settings.

Related papers

• FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning [1.7205106391379026]
  Federated Learning (FL) enables collaborative model training across distributed client devices.<n>We propose FedTVD, a novel FL algorithm that weights client contributions during aggregation by considering both data quality and quantity.<n>We show that FedTVD consistently outperforms state-of-the-art methods across all datasets.
  arXiv  Detail & Related papers  (2026-08-10T07:45:25Z)
• FedLBW: A Loss-Based Weighting Strategy for Federated Learning on Non-IID Data in Wireless Networks [2.4851820343103035]
  Federated Learning (FL) enables collaborative machine learning (ML) across distributed clients while preserving privacy.<n>Traditional FL algorithms, such as FedAvg, rely solely on dataset size to weight client updates.<n>We propose Federated Learning with Loss-Based Weighting (FedLBW), a novel aggregation method that assigns each client's update a weight proportional to the inverse of its validation loss.
  arXiv  Detail & Related papers  (2026-08-07T09:21:37Z)
• Optimized Federated Knowledge Distillation with Distributed Neural Architecture Search [7.461859467262201]
  Federated Learning (FL) enables collaborative model training without centralizing data.<n>Existing FL approaches mitigate these challenges through improved aggregation, personalization, or knowledge distillation.<n>This work introduces FedKDNAS, a distillation-driven FL framework that combines client-side neural architecture selection with distillation of server-coordinated knowledge.
  arXiv  Detail & Related papers  (2026-05-20T15:50:49Z)
• FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Data [0.2840208039211775]
  Federated learning (FL) enables a set of distributed clients to jointly train machine learning models.<n>We propose FedPBS, an FL algorithm that couples complementary ideas from FedBS and FedProx to address these challenges.
  arXiv  Detail & Related papers  (2026-03-14T11:44:27Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout [62.73150122809138]
  Federated Learning (FL) is a promising distributed machine learning approach that enables collaborative training of a global model using multiple edge devices.<n>We propose the FedDHAD FL framework, which comes with two novel methods: Dynamic Heterogeneous model aggregation (FedDH) and Adaptive Dropout (FedAD)<n>The combination of these two methods makes FedDHAD significantly outperform state-of-the-art solutions in terms of accuracy (up to 6.7% higher), efficiency (up to 2.02 times faster), and cost (up to 15.0% smaller)
  arXiv  Detail & Related papers  (2025-07-14T16:19:00Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation [95.85026305874824]
  We introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency.
  arXiv  Detail & Related papers  (2022-12-14T13:57:01Z)
• FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity [56.82825745165945]
  Federated Learning (FL) enables training a global model without sharing the decentralized raw data stored on multiple devices to protect data privacy. We propose a hierarchical synchronous FL framework, i.e., FedHiSyn, to tackle the problems of straggler effects and outdated models. We evaluate the proposed framework based on MNIST, EMNIST, CIFAR10 and CIFAR100 datasets and diverse heterogeneous settings of devices.
  arXiv  Detail & Related papers  (2022-06-21T17:23:06Z)
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning [60.503258658382]
  Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device. Current state-of-the-art algorithms suffer from performance degradation as the heterogeneity of local data across clients increases. We propose a new augmentation algorithm, named FedMix, which is inspired by a phenomenal yet simple data augmentation method, Mixup.
  arXiv  Detail & Related papers  (2021-07-01T06:14:51Z)
• FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measure [18.083188787905083]
  We propose a novel clustered federated learning (CFL) framework FedGroup. We show that FedGroup can significantly improve absolute test accuracy by +14.1% on FEMNIST compared to FedAvg. We also evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets.
  arXiv  Detail & Related papers  (2020-10-14T08:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.