Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms

• URL: http://arxiv.org/abs/2604.20492v1
• Date: Wed, 22 Apr 2026 12:25:09 GMT
• Title: Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms
• Abstract summary: This paper shows for the first time that centralized performance is achievable in decentralized learning without sharing the local datasets.<n>In particular, achieving centralized performance in the decentralized setting requires a specific scaling of the regularization factors with the local sample sizes.
• Score: 3.307203784120635
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, it is shown, for the first time, that centralized performance is achievable in decentralized learning without sharing the local datasets. Specifically, when clients adopt an empirical risk minimization with relative-entropy regularization (ERM-RER) learning framework and a forward-backward communication between clients is established, it suffices to share the locally obtained Gibbs measures to achieve the same performance as that of a centralized ERM-RER with access to all the datasets. The core idea is that the Gibbs measure produced by client~$k$ is used, as reference measure, by client~$k+1$. This effectively establishes a principled way to encode prior information through a reference measure. In particular, achieving centralized performance in the decentralized setting requires a specific scaling of the regularization factors with the local sample sizes. Overall, this result opens the door to novel decentralized learning paradigms that shift the collaboration strategy from sharing data to sharing the local inductive bias via the reference measures over the set of models.

Related papers

• Don't Reach for the Stars: Rethinking Topology for Resilient Federated Learning [1.3270838622986498]
  Federated learning (FL) enables collaborative model training across distributed clients while preserving data privacy by keeping data local.<n>Traditional FL approaches rely on a centralized, star-shaped topology, where a central server aggregates model updates from clients.<n>We propose a decentralized, peer-to-peer (P2P) FL framework to enable each client to identify and aggregate a personalized set of trustworthy and beneficial updates.
  arXiv  Detail & Related papers  (2025-08-07T10:10:37Z)
• Conditioning on Local Statistics for Scalable Heterogeneous Federated Learning [0.0]
  Federated learning is a distributed machine learning approach where multiple clients collaboratively train a model without sharing their local data.<n>We propose to use local characteristic statistics, by which we mean some statistical properties calculated independently by each client.<n>During training, these local statistics help the model learn how to condition on the local data distribution, and during inference, they guide the client's predictions.
  arXiv  Detail & Related papers  (2025-03-01T07:10:58Z)
• Personalized Federated Learning for Cross-view Geo-localization [49.40531019551957]
  We propose a methodology combining Federated Learning (FL) with Cross-view Image Geo-localization (CVGL) techniques. Our method implements a coarse-to-fine approach, where clients share only the coarse feature extractors while keeping fine-grained features specific to local environments. Results demonstrate that our federated CVGL method achieves performance close to centralized training while maintaining data privacy.
  arXiv  Detail & Related papers  (2024-11-07T13:25:52Z)
• Networked Communication for Mean-Field Games with Function Approximation and Empirical Mean-Field Estimation [59.01527054553122]
  Decentralised agents can learn equilibria in Mean-Field Games from a non-episodic run of the empirical system.<n>We introduce function approximation to the existing setting, drawing on the Munchausen Online Mirror Descent method.<n>We show theoretically that exchanging policy information helps networked agents outperform both independent and even centralised agents in function-approximation settings.
  arXiv  Detail & Related papers  (2024-08-21T13:32:46Z)
• Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensus [11.414398732656839]
  We propose a framework that embeds differential privacy into decentralized deep learning and secures each agent's local dataset during and after cooperative training. We prove convergence guarantees for algorithms derived from this framework and demonstrate its practical utility when applied to subgradient and ADMM decentralized approaches.
  arXiv  Detail & Related papers  (2023-06-24T07:46:00Z)
• Collaborative Mean Estimation over Intermittently Connected Networks with Peer-To-Peer Privacy [86.61829236732744]
  This work considers the problem of Distributed Mean Estimation (DME) over networks with intermittent connectivity. The goal is to learn a global statistic over the data samples localized across distributed nodes with the help of a central server. We study the tradeoff between collaborative relaying and privacy leakage due to the additional data sharing among nodes.
  arXiv  Detail & Related papers  (2023-02-28T19:17:03Z)
• Personalized Decentralized Federated Learning with Knowledge Distillation [5.469841541565307]
  Personalization in federated learning functions as a coordinator for clients with high variance in data or behavior. It is generally challenging to quantify similarity under limited knowledge about other users' models given to users in a decentralized network. We propose a personalized and fully decentralized FL algorithm, leveraging knowledge distillation techniques to empower each device so as to discern statistical distances between local models.
  arXiv  Detail & Related papers  (2023-02-23T16:41:07Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Decentralized federated learning of deep neural networks on non-iid data [0.6335848702857039]
  We tackle the non-problem of learning a personalized deep learning model in a decentralized setting. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data detect each other and cooperate. PENS is able to achieve higher accuracies as compared to strong baselines.
  arXiv  Detail & Related papers  (2021-07-18T19:05:44Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)
• Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification [69.29602103582782]
  Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data. However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID) We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients) This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
  arXiv  Detail & Related papers  (2020-06-07T13:32:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.