FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment

• URL: http://arxiv.org/abs/2501.15486v1
• Date: Sun, 26 Jan 2025 11:17:32 GMT
• Title: FedAlign: Federated Domain Generalization with Cross-Client Feature Alignment
• Authors: Sunny Gupta, Vinay Sutar, Varunav Singh, Amit Sethi,
• Abstract summary: Federated Learning (FL) offers a decentralized paradigm for collaborative model training without direct data sharing. It poses unique challenges for Domain Generalization (DG), including strict privacy constraints, non-i.i.d. local data, and limited domain diversity. We introduce FedAlign, a lightweight, privacy-preserving framework designed to enhance DG in federated settings.
• Score: 2.4472081831862655
• License:
• Abstract: Federated Learning (FL) offers a decentralized paradigm for collaborative model training without direct data sharing, yet it poses unique challenges for Domain Generalization (DG), including strict privacy constraints, non-i.i.d. local data, and limited domain diversity. We introduce FedAlign, a lightweight, privacy-preserving framework designed to enhance DG in federated settings by simultaneously increasing feature diversity and promoting domain invariance. First, a cross-client feature extension module broadens local domain representations through domain-invariant feature perturbation and selective cross-client feature transfer, allowing each client to safely access a richer domain space. Second, a dual-stage alignment module refines global feature learning by aligning both feature embeddings and predictions across clients, thereby distilling robust, domain-invariant features. By integrating these modules, our method achieves superior generalization to unseen domains while maintaining data privacy and operating with minimal computational and communication overhead.

Related papers

• FedCCRL: Federated Domain Generalization with Cross-Client Representation Learning [4.703379311088474]
  Domain Generalization (DG) aims to train models that can effectively generalize to unseen domains. In Federated Learning (FL), where clients collaboratively train a model without directly sharing their data, most existing DG algorithms are not directly applicable to the FL setting. We propose FedCCRL, a lightweight federated domain generalization method that significantly improves the model's generalization ability while preserving privacy.
  arXiv  Detail & Related papers  (2024-10-15T04:44:21Z)
• Federated Instruction Tuning of LLMs with Domain Coverage Augmentation [87.49293964617128]
  Federated Domain-specific Instruction Tuning (FedDIT) utilizes limited cross-client private data together with various strategies of instruction augmentation. We propose FedDCA, which optimize domain coverage through greedy client center selection and retrieval-based augmentation. For client-side computational efficiency and system scalability, FedDCA$*$, the variant of FedDCA, utilizes heterogeneous encoders with server-side feature alignment.
  arXiv  Detail & Related papers  (2024-09-30T09:34:31Z)
• FedGCA: Global Consistent Augmentation Based Single-Source Federated Domain Generalization [29.989092118578103]
  Federated Domain Generalization (FedDG) aims to train the global model for generalization ability to unseen domains with multi-domain training samples. Clients in federated learning networks are often confined to a single, non-IID domain due to inherent sampling and temporal limitations. We introduce the Federated Global Consistent Augmentation (FedGCA) method, which incorporates a style-complement module to augment data samples with diverse domain styles.
  arXiv  Detail & Related papers  (2024-09-23T02:24:46Z)
• Feature Diversification and Adaptation for Federated Domain Generalization [27.646565383214227]
  In real-world applications, local clients often operate within their limited domains, leading to a domain shift' across clients. We introduce the concept of federated feature diversification, which helps local models learn client-invariant representations while preserving privacy. Our resultant global model shows robust performance on unseen test domain data.
  arXiv  Detail & Related papers  (2024-07-11T07:45:10Z)
• Hypernetwork-Driven Model Fusion for Federated Domain Generalization [26.492360039272942]
  Federated Learning (FL) faces significant challenges with domain shifts in heterogeneous data. We propose a robust framework, coined as hypernetwork-based Federated Fusion (hFedF), using hypernetworks for non-linear aggregation. Our method employs client-specific embeddings and gradient alignment techniques to manage domain generalization effectively.
  arXiv  Detail & Related papers  (2024-02-10T15:42:03Z)
• FACT: Federated Adversarial Cross Training [0.0]
  Federated Adrial Cross Training (FACT) uses implicit domain differences between source clients to identify domain shifts in the target domain. We empirically show that FACT outperforms state-of-the-art federated, non-federated and source-free domain adaptation models.
  arXiv  Detail & Related papers  (2023-06-01T12:25:43Z)
• Compound Domain Generalization via Meta-Knowledge Encoding [55.22920476224671]
  We introduce Style-induced Domain-specific Normalization (SDNorm) to re-normalize the multi-modal underlying distributions. We harness the prototype representations, the centroids of classes, to perform relational modeling in the embedding space. Experiments on four standard Domain Generalization benchmarks reveal that COMEN exceeds the state-of-the-art performance without the need of domain supervision.
  arXiv  Detail & Related papers  (2022-03-24T11:54:59Z)
• TAL: Two-stream Adaptive Learning for Generalizable Person Re-identification [115.31432027711202]
  We argue that both domain-specific and domain-invariant features are crucial for improving the generalization ability of re-id models. We name two-stream adaptive learning (TAL) to simultaneously model these two kinds of information. Our framework can be applied to both single-source and multi-source domain generalization tasks.
  arXiv  Detail & Related papers  (2021-11-29T01:27:42Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Cross-Domain Grouping and Alignment for Domain Adaptive Semantic Segmentation [74.3349233035632]
  Existing techniques to adapt semantic segmentation networks across the source and target domains within deep convolutional neural networks (CNNs) do not consider an inter-class variation within the target domain itself or estimated category. We introduce a learnable clustering module, and a novel domain adaptation framework called cross-domain grouping and alignment. Our method consistently boosts the adaptation performance in semantic segmentation, outperforming the state-of-the-arts on various domain adaptation settings.
  arXiv  Detail & Related papers  (2020-12-15T11:36:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.