Deep Domain Isolation and Sample Clustered Federated Learning for Semantic Segmentation

• URL: http://arxiv.org/abs/2410.14693v1
• Date: Fri, 04 Oct 2024 12:43:07 GMT
• Title: Deep Domain Isolation and Sample Clustered Federated Learning for Semantic Segmentation
• Authors: Matthis Manthe, Carole Lartizien, Stefan Duffner,
• Abstract summary: In this paper, we explore for the first time the effect of covariate shifts between participants' data in 2D segmentation tasks. We develop Deep Domain Isolation (DDI) to isolate image domains directly in the gradient space of the model. We leverage this clustering algorithm through a Sample Clustered Federated Learning (SCFL) framework.
• Score: 2.515027627030043
• License:
• Abstract: Empirical studies show that federated learning exhibits convergence issues in Non Independent and Identically Distributed (IID) setups. However, these studies only focus on label distribution shifts, or concept shifts (e.g. ambiguous tasks). In this paper, we explore for the first time the effect of covariate shifts between participants' data in 2D segmentation tasks, showing an impact way less serious than label shifts but still present on convergence. Moreover, current Personalized (PFL) and Clustered (CFL) Federated Learning methods intrinsically assume the homogeneity of the dataset of each participant and its consistency with future test samples by operating at the client level. We introduce a more general and realistic framework where each participant owns a mixture of multiple underlying feature domain distributions. To diagnose such pathological feature distributions affecting a model being trained in a federated fashion, we develop Deep Domain Isolation (DDI) to isolate image domains directly in the gradient space of the model. A federated Gaussian Mixture Model is fit to the sample gradients of each class, while the results are combined with spectral clustering on the server side to isolate decentralized sample-level domains. We leverage this clustering algorithm through a Sample Clustered Federated Learning (SCFL) framework, performing standard federated learning of several independent models, one for each decentralized image domain. Finally, we train a classifier enabling to associate a test sample to its corresponding domain cluster at inference time, offering a final set of models that are agnostic to any assumptions on the test distribution of each participant. We validate our approach on a toy segmentation dataset as well as different partitionings of a combination of Cityscapes and GTA5 datasets using an EfficientVIT-B0 model, showing a significant performance gain compared to other approaches. Our code is available at https://github.com/MatthisManthe/DDI_SCFL .

Related papers

• Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains [8.047147770476212]
  Federated learning (FL) allows collaborative machine learning training without sharing private data. While most FL methods assume identical data domains across clients, real-world scenarios often involve heterogeneous data domains. We introduce FedPLVM, which establishes variance-aware dual-level prototypes clustering and employs a novel $alpha$-sparsity prototype loss.
  arXiv  Detail & Related papers  (2024-03-14T02:36:16Z)
• CNG-SFDA:Clean-and-Noisy Region Guided Online-Offline Source-Free Domain Adaptation [6.222371087167951]
  Source-Free Domain Adaptation (SFDA) aims to adopt a trained model on the source domain to the target domain. handling false labels in the target domain is crucial because they negatively impact the model performance. We conduct extensive experiments on multiple datasets in online/offline SFDA settings, whose results demonstrate that our method, CNG-SFDA, achieves state-of-the-art for most cases.
  arXiv  Detail & Related papers  (2024-01-26T01:29:37Z)
• Federated Two Stage Decoupling With Adaptive Personalization Layers [5.69361786082969]
  Federated learning has gained significant attention due to its ability to enable distributed learning while maintaining privacy constraints. It inherently experiences significant learning degradation and slow convergence speed. It is natural to employ the concept of clustering homogeneous clients into the same group, allowing only the model weights within each group to be aggregated.
  arXiv  Detail & Related papers  (2023-08-30T07:46:32Z)
• Compositional Semantic Mix for Domain Adaptation in Point Cloud Segmentation [65.78246406460305]
  compositional semantic mixing represents the first unsupervised domain adaptation technique for point cloud segmentation. We present a two-branch symmetric network architecture capable of concurrently processing point clouds from a source domain (e.g. synthetic) and point clouds from a target domain (e.g. real-world)
  arXiv  Detail & Related papers  (2023-08-28T14:43:36Z)
• Generalizable Metric Network for Cross-domain Person Re-identification [55.71632958027289]
  Cross-domain (i.e., domain generalization) scene presents a challenge in Re-ID tasks. Most existing methods aim to learn domain-invariant or robust features for all domains. We propose a Generalizable Metric Network (GMN) to explore sample similarity in the sample-pair space.
  arXiv  Detail & Related papers  (2023-06-21T03:05:25Z)
• Bidirectional Domain Mixup for Domain Adaptive Semantic Segmentation [73.3083304858763]
  This paper systematically studies the impact of mixup under the domain adaptaive semantic segmentation task. In specific, we achieve domain mixup in two-step: cut and paste. We provide extensive ablation experiments to empirically verify our main components of the framework.
  arXiv  Detail & Related papers  (2023-03-17T05:22:44Z)
• Synthetic-to-Real Domain Generalized Semantic Segmentation for 3D Indoor Point Clouds [69.64240235315864]
  This paper introduces the synthetic-to-real domain generalization setting to this task. The domain gap between synthetic and real-world point cloud data mainly lies in the different layouts and point patterns. Experiments on the synthetic-to-real benchmark demonstrate that both CINMix and multi-prototypes can narrow the distribution gap.
  arXiv  Detail & Related papers  (2022-12-09T05:07:43Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data [37.667379000751325]
  Federated learning (FL) is a distributed learning method that offers medical institutes the prospect of collaboration in a global model. In this work, we investigate an adaptive hierarchical clustering method for FL to produce intermediate semi-global models. Our experiments demonstrate significant performance gain in heterogeneous distribution compared to standard FL methods in classification accuracy.
  arXiv  Detail & Related papers  (2022-07-07T17:25:04Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.