FedBEns: One-Shot Federated Learning based on Bayesian Ensemble

• URL: http://arxiv.org/abs/2503.15367v1
• Date: Wed, 19 Mar 2025 16:05:52 GMT
• Title: FedBEns: One-Shot Federated Learning based on Bayesian Ensemble
• Authors: Jacopo Talpini, Marco Savi, Giovanni Neglia,
• Abstract summary: One-Shot Federated Learning (FL) is a recent paradigm that enables multiple clients to cooperatively learn a global model in a single round of communication with a central server.<n>We propose FedBEns, an algorithm that leverages the inherent multimodality of local loss functions to find better global models.
• Score: 9.653976364051564
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: One-Shot Federated Learning (FL) is a recent paradigm that enables multiple clients to cooperatively learn a global model in a single round of communication with a central server. In this paper, we analyze the One-Shot FL problem through the lens of Bayesian inference and propose FedBEns, an algorithm that leverages the inherent multimodality of local loss functions to find better global models. Our algorithm leverages a mixture of Laplace approximations for the clients' local posteriors, which the server then aggregates to infer the global model. We conduct extensive experiments on various datasets, demonstrating that the proposed method outperforms competing baselines that typically rely on unimodal approximations of the local losses.

Related papers

• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• FedHB: Hierarchical Bayesian Federated Learning [11.936836827864095]
  We propose a novel hierarchical Bayesian approach to Federated Learning (FL) Our model reasonably describes the generative process of clients' local data via hierarchical Bayesian modeling. We show that our block-coordinate FL algorithm converges to an optimum of the objective at the rate of $O(sqrtt)$.
  arXiv  Detail & Related papers  (2023-05-08T18:21:41Z)
• Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach [66.9033666087719]
  This paper extends the inference view and describes a variational inference formulation of federated learning. We apply FedEP on standard federated learning benchmarks and find that it outperforms strong baselines in terms of both convergence speed and accuracy.
  arXiv  Detail & Related papers  (2023-02-08T17:58:11Z)
• Personalized Federated Learning with Hidden Information on Personalized Prior [18.8426865970643]
  We propose pFedBreD, a framework to solve the problem we model using Bregman divergence regularization. Our experiments show that our proposal significantly outcompetes other PFL algorithms on multiple public benchmarks.
  arXiv  Detail & Related papers  (2022-11-19T12:45:19Z)
• Federated Multi-Task Learning under a Mixture of Distributions [10.00087964926414]
  Federated Learning (FL) is a framework for on-device collaborative training of machine learning models. First efforts in FL focused on learning a single global model with good average performance across clients, but the global model may be arbitrarily bad for a given client. We study federated MTL under the flexible assumption that each local data distribution is a mixture of unknown underlying distributions.
  arXiv  Detail & Related papers  (2021-08-23T15:47:53Z)
• A Bayesian Federated Learning Framework with Online Laplace Approximation [144.7345013348257]
  Federated learning allows multiple clients to collaboratively learn a globally shared model. We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side. We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
  arXiv  Detail & Related papers  (2021-02-03T08:36:58Z)
• Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms [21.11885845002748]
  We present an alternative perspective and formulate federated learning as a posterior inference problem. The goal is to infer a global posterior distribution by having client devices each infer the posterior of their local data. While exact inference is often intractable, this perspective provides a principled way to search for global optima in federated settings.
  arXiv  Detail & Related papers  (2020-10-11T15:55:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.