Zero-Shot Decentralized Federated Learning

• URL: http://arxiv.org/abs/2509.26462v1
• Date: Tue, 30 Sep 2025 16:13:21 GMT
• Title: Zero-Shot Decentralized Federated Learning
• Abstract summary: We propose Zero-shot Decentralized Federated Learning (ZeroDFL), a fully decentralized framework that enables zero-shot adaptation across distributed clients without a central coordinator.<n>We validate ZeroDFL on nine diverse image classification datasets, demonstrating that it consistently outperforms--or remains on par with--state-of-the-art federated prompt learning methods.<n>These results highlight that our approach not only enhances generalization in federated zero-shot learning but also improves scalability, efficiency, and privacy preservation--paving the way for decentralized adaptation of large vision-language models in real-world applications.
• Score: 10.60647737661699
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: CLIP has revolutionized zero-shot learning by enabling task generalization without fine-tuning. While prompting techniques like CoOp and CoCoOp enhance CLIP's adaptability, their effectiveness in Federated Learning (FL) remains an open challenge. Existing federated prompt learning approaches, such as FedCoOp and FedTPG, improve performance but face generalization issues, high communication costs, and reliance on a central server, limiting scalability and privacy. We propose Zero-shot Decentralized Federated Learning (ZeroDFL), a fully decentralized framework that enables zero-shot adaptation across distributed clients without a central coordinator. ZeroDFL employs an iterative prompt-sharing mechanism, allowing clients to optimize and exchange textual prompts to enhance generalization while drastically reducing communication overhead. We validate ZeroDFL on nine diverse image classification datasets, demonstrating that it consistently outperforms--or remains on par with--state-of-the-art federated prompt learning methods. More importantly, ZeroDFL achieves this performance in a fully decentralized setting while reducing communication overhead by 118x compared to FedTPG. These results highlight that our approach not only enhances generalization in federated zero-shot learning but also improves scalability, efficiency, and privacy preservation--paving the way for decentralized adaptation of large vision-language models in real-world applications.

Related papers

• A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints [12.544579930854447]
  Decentralized learning systems aim to collaboratively train models across multiple clients without relying on a central coordinator.<n>While decentralization improves scalability, privacy, and robustness, it also exacerbates three fundamental challenges: statistical heterogeneity across clients, synchronization in client-level performance, and stringent communication constraints.<n>Specifically, we propose a new algorithm DMFL-SQ, a decentralized multi-task learning algorithm that agnostic personalized model over a communication graph with an objective, while reducing sparsification through quantization.<n>We show that DMFL-SQ substantially reduces communication while maintaining predictive performance and improving fairness across clients.
  arXiv  Detail & Related papers  (2026-08-27T00:28:29Z)
• Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching [19.959825317183924]
  Federated Learning (FL) enables collaborative training across decentralized data.<n>We propose pFed1BS, a novel personalized FL framework that achieves extreme communication compression through one-bit random sketching.<n>In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus.
  arXiv  Detail & Related papers  (2025-11-17T08:55:22Z)
• On the Fast Adaptation of Delayed Clients in Decentralized Federated Learning: A Centroid-Aligned Distillation Approach [0.27309692684728615]
  DFedCAD is a novel framework for rapid adaptation via Centroid-Aligned Distillation.<n>It enables delayed clients to intelligently weigh and align with peer knowledge.<n>Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that DFedCAD consistently achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-08-05T01:41:30Z)
• A Single Merging Suffices: Recovering Server-based Learning Performance in Decentralized Learning [17.386971981099588]
  We study how communication should be scheduled over time, including determining when and how frequently devices synchronize.<n>We find that fully connected communication at the final step, implemented by a single global merging, is sufficient to match the performance of server-based training.
  arXiv  Detail & Related papers  (2025-07-09T04:56:56Z)
• UA-PDFL: A Personalized Approach for Decentralized Federated Learning [5.065947993017158]
  Federated learning (FL) is a privacy preserving machine learning paradigm designed to collaboratively learn a global model without data leakage.<n>To mitigate this issue, decentralized federated learning (DFL) has been proposed, where all participating clients engage in peer-to-peer communication without a central server.<n>We propose a novel unit representation aided personalized decentralized federated learning framework, named UA-PDFL, to deal with the non-IID challenge in DFL.
  arXiv  Detail & Related papers  (2024-12-16T11:27:35Z)
• Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration [66.43954501171292]
  We introduce Catalyst Acceleration and propose an acceleration Decentralized Federated Learning algorithm called DFedCata. DFedCata consists of two main components: the Moreau envelope function, which addresses parameter inconsistencies, and Nesterov's extrapolation step, which accelerates the aggregation phase. Empirically, we demonstrate the advantages of the proposed algorithm in both convergence speed and generalization performance on CIFAR10/100 with various non-iid data distributions.
  arXiv  Detail & Related papers  (2024-10-09T06:17:16Z)
• SCALE: Self-regulated Clustered federAted LEarning in a Homogeneous Environment [4.925906256430176]
  Federated Learning (FL) has emerged as a transformative approach for enabling distributed machine learning while preserving user privacy. This paper presents a novel FL methodology that overcomes these limitations by eliminating the dependency on edge servers.
  arXiv  Detail & Related papers  (2024-07-25T20:42:16Z)
• FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models [52.13056951033747]
  Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server.<n>A critical bottleneck in FL is the communication cost.<n>Our work is inspired by the innovative Scaffnew algorithm, which has advanced the reduction of communication complexity in FL.<n>We introduce FedComLoc, integrating practical and effective compression into Scaffnew to further enhance communication efficiency.
  arXiv  Detail & Related papers  (2024-03-14T22:29:59Z)
• Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization [71.87335804334616]
  Federated learning (FL) is a promising paradigm to enable collaborative model training with decentralized data. The training process of Large Language Models (LLMs) generally incurs the update of significant parameters. This paper proposes an efficient partial prompt tuning approach to improve performance and efficiency simultaneously.
  arXiv  Detail & Related papers  (2023-10-23T16:37:59Z)
• Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training [67.67045085186797]
  Almost all existing systems have to face large communication burdens if the central FL server fails. It personalizes the "right" in the deep models by alternately updating the shared and personal parameters. To further promote the shared parameters aggregation process, we propose DFed integrating the local Sharpness Miniization.
  arXiv  Detail & Related papers  (2023-05-24T13:52:18Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping [77.53019031244908]
  We present a novel decentralized learning framework based on nonlinear gossiping (NGO), that enjoys an appealing finite-time consensus property to achieve better synchronization. Our analysis on how communication delay and randomized chats affect learning further enables the derivation of practical variants.
  arXiv  Detail & Related papers  (2021-11-04T15:36:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.