Collaborative Batch Size Optimization for Federated Learning

• URL: http://arxiv.org/abs/2506.20511v1
• Date: Wed, 25 Jun 2025 14:57:23 GMT
• Title: Collaborative Batch Size Optimization for Federated Learning
• Authors: Arno Geimer, Karthick Panner Selvam, Beltran Fiz Pontiveros,
• Abstract summary: This paper focuses on improving the local training process through hardware usage optimization.<n>Taking advantage of the parallel processing inherent to Federated Learning, we use a greedy randomized search to optimize local batch sizes.<n>Our results show that against default parameter settings, our method improves convergence speed while staying nearly on par with the case where local parameters are optimized.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) is a decentralized collaborative Machine Learning framework for training models without collecting data in a centralized location. It has seen application across various disciplines, from helping medical diagnoses in hospitals to detecting fraud in financial transactions. In this paper, we focus on improving the local training process through hardware usage optimization. While participants in a federation might share the hardware they are training on, since there is no information exchange between them, their training process can be hindered by an improper training configuration. Taking advantage of the parallel processing inherent to Federated Learning, we use a greedy randomized search to optimize local batch sizes for the best training settings across all participants. Our results show that against default parameter settings, our method improves convergence speed while staying nearly on par with the case where local parameters are optimized.

Related papers

• PFedDST: Personalized Federated Learning with Decentralized Selection Training [8.21688083335571]
  We introduce the Personalized Federated Learning with Decentralized Selection Training (PFedDST) framework.<n>PFedDST enhances model training by allowing devices to strategically evaluate and select peers based on a comprehensive communication score.<n>Our experiments demonstrate that PFedDST not only enhances model accuracy but also accelerates convergence.
  arXiv  Detail & Related papers  (2025-02-11T18:25:48Z)
• Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learning [56.21666819468249]
  In Federated Learning (FL), both client resource constraints and communication costs pose major problems for training large models.<n>Recent work has shown that local training provably improves communication complexity through acceleration.<n>We introduce Sparse-ProxSkip, addressing the issue and implementing the efficient technique of Straight-Through Estorimat pruning into sparse training.
  arXiv  Detail & Related papers  (2024-05-31T05:21:12Z)
• Federated Bayesian Network Ensembles [3.24530181403525]
  Federated learning allows us to run machine learning algorithms on decentralized data when data sharing is not permitted due to privacy concerns. We show that FBNE is a potentially useful tool within the federated learning toolbox. We discuss the advantages and disadvantages of this approach in terms of time complexity, model accuracy, privacy protection, and model interpretability.
  arXiv  Detail & Related papers  (2024-02-19T13:52:37Z)
• FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data [54.81695390763957]
  Federated learning is an emerging distributed machine learning method. We propose a heterogeneous local variant of AMSGrad, named FedLALR, in which each client adjusts its learning rate. We show that our client-specified auto-tuned learning rate scheduling can converge and achieve linear speedup with respect to the number of clients.
  arXiv  Detail & Related papers  (2023-09-18T12:35:05Z)
• TAMUNA: Doubly Accelerated Distributed Optimization with Local Training, Compression, and Partial Participation [53.84175614198885]
  In distributed optimization and learning, several machines alternate between local computations in parallel and communication with a distant server. We propose TAMUNA, the first algorithm for distributed optimization that leveraged the two strategies of local training and compression jointly and allows for partial participation.
  arXiv  Detail & Related papers  (2023-02-20T08:37:44Z)
• Certified Robustness in Federated Learning [54.03574895808258]
  We study the interplay between federated training, personalization, and certified robustness. We find that the simple federated averaging technique is effective in building not only more accurate, but also more certifiably-robust models.
  arXiv  Detail & Related papers  (2022-06-06T12:10:53Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• On Second-order Optimization Methods for Federated Learning [59.787198516188425]
  We evaluate the performance of several second-order distributed methods with local steps in the federated learning setting. We propose a novel variant that uses second-order local information for updates and a global line search to counteract the resulting local specificity.
  arXiv  Detail & Related papers  (2021-09-06T12:04:08Z)
• Local Adaptivity in Federated Learning: Convergence and Consistency [25.293584783673413]
  Federated learning (FL) framework trains a machine learning model using decentralized data stored at edge client devices by periodically aggregating locally trained models. We show in both theory and practice that while local adaptive methods can accelerate convergence, they can cause a non-vanishing solution bias. We propose correction techniques to overcome this inconsistency and complement the local adaptive methods for FL.
  arXiv  Detail & Related papers  (2021-06-04T07:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.