Delta Sum Learning: an approach for fast and global convergence in Gossip Learning

• URL: http://arxiv.org/abs/2512.01549v1
• Date: Mon, 01 Dec 2025 11:23:51 GMT
• Title: Delta Sum Learning: an approach for fast and global convergence in Gossip Learning
• Authors: Tom Goethals, Merlijn Sebrechts, Stijn De Schrijver, Filip De Turck, Bruno Volckaert,
• Abstract summary: Gossip Learning further decentralizes Federated Learning by removing centralized integration and relying fully on peer to peer updates.<n>The averaging methods generally used in both Federated and Gossip Learning are not ideal for model accuracy and global convergence.<n>This paper proposes Delta Sum Learning as a method to improve the basic aggregation operation in Gossip Learning.
• Score: 2.617654879276715
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Federated Learning is a popular approach for distributed learning due to its security and computational benefits. With the advent of powerful devices in the network edge, Gossip Learning further decentralizes Federated Learning by removing centralized integration and relying fully on peer to peer updates. However, the averaging methods generally used in both Federated and Gossip Learning are not ideal for model accuracy and global convergence. Additionally, there are few options to deploy Learning workloads in the edge as part of a larger application using a declarative approach such as Kubernetes manifests. This paper proposes Delta Sum Learning as a method to improve the basic aggregation operation in Gossip Learning, and implements it in a decentralized orchestration framework based on Open Application Model, which allows for dynamic node discovery and intent-driven deployment of multi-workload applications. Evaluation results show that Delta Sum performance is on par with alternative integration methods for 10 node topologies, but results in a 58% lower global accuracy drop when scaling to 50 nodes. Overall, it shows strong global convergence and a logarithmic loss of accuracy with increasing topology size compared to a linear loss for alternatives under limited connectivity.

Related papers

• Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration [66.43954501171292]
  We introduce Catalyst Acceleration and propose an acceleration Decentralized Federated Learning algorithm called DFedCata. DFedCata consists of two main components: the Moreau envelope function, which addresses parameter inconsistencies, and Nesterov's extrapolation step, which accelerates the aggregation phase. Empirically, we demonstrate the advantages of the proposed algorithm in both convergence speed and generalization performance on CIFAR10/100 with various non-iid data distributions.
  arXiv  Detail & Related papers  (2024-10-09T06:17:16Z)
• Scheduling and Communication Schemes for Decentralized Federated Learning [0.31410859223862103]
  A decentralized federated learning (DFL) model with the gradient descent (SGD) algorithm has been introduced. Three scheduling policies for DFL have been proposed for communications between the clients and the parallel servers. Results show that the proposed scheduling polices have an impact both on the speed of convergence and in the final global model.
  arXiv  Detail & Related papers  (2023-11-27T17:35:28Z)
• Sparse Decentralized Federated Learning [35.32297764027417]
  Decentralized Federated Learning (DFL) enables collaborative model training without a central server but faces challenges in efficiency, stability, and trustworthiness.<n>We introduce a sparsity constraint on the shared model, leading to Sparse DFL (SDFL), and propose a novel algorithm, CEPS.<n> Numerical experiments validate the effectiveness of the proposed algorithm in improving communication and efficiency while maintaining a high level of trustworthiness.
  arXiv  Detail & Related papers  (2023-08-31T12:22:40Z)
• Stochastic Unrolled Federated Learning [85.6993263983062]
  We introduce UnRolled Federated learning (SURF), a method that expands algorithm unrolling to federated learning. Our proposed method tackles two challenges of this expansion, namely the need to feed whole datasets to the unrolleds and the decentralized nature of federated learning.
  arXiv  Detail & Related papers  (2023-05-24T17:26:22Z)
• Distributed Learning over Networks with Graph-Attention-Based Personalization [49.90052709285814]
  We propose a graph-based personalized algorithm (GATTA) for distributed deep learning. In particular, the personalized model in each agent is composed of a global part and a node-specific part. By treating each agent as one node in a graph the node-specific parameters as its features, the benefits of the graph attention mechanism can be inherited.
  arXiv  Detail & Related papers  (2023-05-22T13:48:30Z)
• Recursive Euclidean Distance Based Robust Aggregation Technique For Federated Learning [4.848016645393023]
  Federated learning is a solution to data availability and privacy challenges in machine learning. Malicious users aim to sabotage the collaborative learning process by training the local model with malicious data. We propose a novel robust aggregation approach based on Euclidean distance calculation.
  arXiv  Detail & Related papers  (2023-03-20T06:48:43Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach [66.9033666087719]
  This paper extends the inference view and describes a variational inference formulation of federated learning. We apply FedEP on standard federated learning benchmarks and find that it outperforms strong baselines in terms of both convergence speed and accuracy.
  arXiv  Detail & Related papers  (2023-02-08T17:58:11Z)
• Server Averaging for Federated Learning [14.846231685735592]
  Federated learning allows distributed devices to collectively train a model without sharing or disclosing the local dataset with a central server. The improved privacy of federated learning also introduces challenges including higher computation and communication costs. We propose the server averaging algorithm to accelerate convergence.
  arXiv  Detail & Related papers  (2021-03-22T07:07:00Z)
• BayGo: Joint Bayesian Learning and Information-Aware Graph Optimization [48.30183416069897]
  BayGo is a novel fully decentralized joint Bayesian learning and graph optimization framework. We show that our framework achieves faster convergence and higher accuracy compared to fully-connected and star topology graphs.
  arXiv  Detail & Related papers  (2020-11-09T11:16:55Z)
• Jointly Optimizing Dataset Size and Local Updates in Heterogeneous Mobile Edge Learning [11.191719032853527]
  This paper proposes to maximize the accuracy of a distributed machine learning (ML) model trained on learners connected via the resource-constrained wireless edge. We jointly optimize the number of local/global updates and the task size allocation to minimize the loss while taking into account heterogeneous communication and computation capabilities of each learner.
  arXiv  Detail & Related papers  (2020-06-12T18:19:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.