Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting

• URL: http://arxiv.org/abs/2302.05049v4
• Date: Sun, 24 Mar 2024 22:00:52 GMT
• Title: Principled Federated Domain Adaptation: Gradient Projection and Auto-Weighting
• Abstract summary: Federated Domain Adaptation (FDA) describes the federated learning setting where source clients and a server work collaboratively to improve the performance of a target client where limited data is available. We introduce new metrics that characterize FDA setting and a theoretical framework with novel theorems for analyzing the performance of server aggregation rules. We propose a novel lightweight aggregation rule, Federated Gradient Projection ($texttFedGP$), which significantly improves the target performance with domain shift and data scarcity.
• Score: 12.600755361120374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Domain Adaptation (FDA) describes the federated learning (FL) setting where source clients and a server work collaboratively to improve the performance of a target client where limited data is available. The domain shift between the source and target domains, coupled with limited data of the target client, makes FDA a challenging problem, e.g., common techniques such as federated averaging and fine-tuning fail due to domain shift and data scarcity. To theoretically understand the problem, we introduce new metrics that characterize the FDA setting and a theoretical framework with novel theorems for analyzing the performance of server aggregation rules. Further, we propose a novel lightweight aggregation rule, Federated Gradient Projection ($\texttt{FedGP}$), which significantly improves the target performance with domain shift and data scarcity. Moreover, our theory suggests an $\textit{auto-weighting scheme}$ that finds the optimal combinations of the source and target gradients. This scheme improves both $\texttt{FedGP}$ and a simpler heuristic aggregation rule. Extensive experiments verify the theoretical insights and illustrate the effectiveness of the proposed methods in practice.

Related papers

• FeDecider: An LLM-Based Framework for Federated Cross-Domain Recommendation [75.50721642765994]
  Large language model (LLM)-based recommendation models have demonstrated impressive performance.<n>We propose an LLM-based framework for Federated cross-domain recommendation, FeDecider.<n>Extensive experiments across diverse datasets validate the effectiveness of our proposed FeDecider.
  arXiv  Detail & Related papers  (2026-02-17T21:42:28Z)
• Adaptive Weighted Loss for Sequential Recommendations on Sparse Domains [0.0]
  A single, uniform weight may not be sufficient for domains with very few interactions, where the training signal is easily diluted by the vast, generic dataset.<n>This paper proposes a novel, data-driven approach: a Dynamic Weighted Loss function with comprehensive theoretical foundations and extensive empirical validation.
  arXiv  Detail & Related papers  (2025-10-05T21:42:33Z)
• Rethinking the Backbone in Class Imbalanced Federated Source Free Domain Adaptation: The Utility of Vision Foundation Models [9.432150710329609]
  Federated Learning (FL) offers a framework for training models collaboratively while preserving data privacy of each client.<n>We propose replacing the FFREEDA backbone with a frozen vision foundation model (VFM)<n>VFMs effectively mitigate the effects of domain gaps, class imbalances, and even non-IID-ness among target clients.
  arXiv  Detail & Related papers  (2025-09-10T08:10:57Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture. Non-smooth regularization is often incorporated into machine learning tasks. We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning [15.640664498531274]
  Federated Domain Adaptation (FDA) is a Federated Learning (FL) scenario where models are trained across multiple clients without transmitting private data. We propose a novel framework called textbfMulti-domain textbfPrototype-based textbfFederated Fine-textbfTuning (MPFT) MPFT fine-tunes a pre-trained model using multi-domain prototypes, i.e., pretrained representations enriched with domain-specific information from category-specific local data.
  arXiv  Detail & Related papers  (2024-10-10T09:15:56Z)
• Efficiently Assemble Normalization Layers and Regularization for Federated Domain Generalization [1.1534313664323637]
  Domain shift is a formidable issue in Machine Learning that causes a model to suffer from performance degradation when tested on unseen domains. FedDG attempts to train a global model using collaborative clients in a privacy-preserving manner that can generalize well to unseen clients possibly with domain shift. Here, we introduce a novel architectural method for FedDG, namely gPerXAN, which relies on a normalization scheme working with a guiding regularizer.
  arXiv  Detail & Related papers  (2024-03-22T20:22:08Z)
• Hypernetwork-Driven Model Fusion for Federated Domain Generalization [26.492360039272942]
  Federated Learning (FL) faces significant challenges with domain shifts in heterogeneous data. We propose a robust framework, coined as hypernetwork-based Federated Fusion (hFedF), using hypernetworks for non-linear aggregation. Our method employs client-specific embeddings and gradient alignment techniques to manage domain generalization effectively.
  arXiv  Detail & Related papers  (2024-02-10T15:42:03Z)
• Model-Contrastive Federated Domain Adaptation [3.9435648520559177]
  Federated domain adaptation (FDA) aims to collaboratively transfer knowledge from source clients (domains) to the related but different target client. We propose a model-based method named FDAC, aiming to address bf Federated bf Domain bf Adaptation based on bf Contrastive learning and Vision Transformer (ViT) To the best of our knowledge, FDAC is the first attempt to learn transferable representations by manipulating the latent architecture of ViT under the federated setting.
  arXiv  Detail & Related papers  (2023-05-07T23:48:03Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• AdaTriplet-RA: Domain Matching via Adaptive Triplet and Reinforced Attention for Unsupervised Domain Adaptation [15.905869933337101]
  Unsupervised domain adaption (UDA) is a transfer learning task where the data and annotations of the source domain are available but only have access to the unlabeled target data during training. We propose to improve the unsupervised domain adaptation task with an inter-domain sample matching scheme. We apply the widely-used and robust Triplet loss to match the inter-domain samples. To reduce the catastrophic effect of the inaccurate pseudo-labels generated during training, we propose a novel uncertainty measurement method to select reliable pseudo-labels automatically and progressively refine them.
  arXiv  Detail & Related papers  (2022-11-16T13:04:24Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection [79.89082006155135]
  Unsupervised Domain Adaptation (UDA) is an effective approach to tackle the issue of domain shift. UDA methods try to align the source and target representations to improve the generalization on the target domain. The Source-Free Adaptation Domain (SFDA) setting aims to alleviate these concerns by adapting a source-trained model for the target domain without requiring access to the source data.
  arXiv  Detail & Related papers  (2022-03-29T17:50:43Z)
• FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation [6.3434032890855345]
  Federated domain adaptation is a unique minimax training task due to the prevalence of label imbalance among clients. We propose a distributed minimax domain referred to as FedMM, designed specifically for the federated adversary adaptation problem. We prove that FedMM ensures convergence to a stationary point with domain-shifted unsupervised data.
  arXiv  Detail & Related papers  (2021-10-16T05:32:03Z)
• Faster Non-Convex Federated Learning via Global and Local Momentum [57.52663209739171]
  textttFedGLOMO is the first (first-order) FLtexttFedGLOMO algorithm. Our algorithm is provably optimal even with communication between the clients and the server.
  arXiv  Detail & Related papers  (2020-12-07T21:05:31Z)
• Learning Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation with Few Labeled Source Samples [65.55521019202557]
  Domain adaptation aims to generalize a model from a source domain to tackle tasks in a related but different target domain. Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain. We propose a Domain-invariant Graph Learning (DGL) approach for domain adaptation with only a few labeled source samples.
  arXiv  Detail & Related papers  (2020-08-21T08:13:25Z)
• Supervised Domain Adaptation using Graph Embedding [86.3361797111839]
  Domain adaptation methods assume that distributions between the two domains are shifted and attempt to realign them. We propose a generic framework based on graph embedding. We show that the proposed approach leads to a powerful Domain Adaptation framework.
  arXiv  Detail & Related papers  (2020-03-09T12:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.