MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligence

• URL: http://arxiv.org/abs/2304.01950v2
• Date: Wed, 11 Oct 2023 14:21:29 GMT
• Title: MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligence
• Abstract summary: This paper proposes a multi-prototype federated contrastive learning approach (MP-FedCL) It demonstrates the effectiveness of using a multi-prototype strategy over a single-prototype under non-IID settings. Experimental results on MNIST, Digit-5, Office-10, and DomainNet show that our method outperforms multiple baselines.
• Score: 41.75096257083503
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Federated learning-assisted edge intelligence enables privacy protection in modern intelligent services. However, not independent and identically distributed (non-IID) distribution among edge clients can impair the local model performance. The existing single prototype-based strategy represents a class by using the mean of the feature space. However, feature spaces are usually not clustered, and a single prototype may not represent a class well. Motivated by this, this paper proposes a multi-prototype federated contrastive learning approach (MP-FedCL) which demonstrates the effectiveness of using a multi-prototype strategy over a single-prototype under non-IID settings, including both label and feature skewness. Specifically, a multi-prototype computation strategy based on \textit{k-means} is first proposed to capture different embedding representations for each class space, using multiple prototypes ($k$ centroids) to represent a class in the embedding space. In each global round, the computed multiple prototypes and their respective model parameters are sent to the edge server for aggregation into a global prototype pool, which is then sent back to all clients to guide their local training. Finally, local training for each client minimizes their own supervised learning tasks and learns from shared prototypes in the global prototype pool through supervised contrastive learning, which encourages them to learn knowledge related to their own class from others and reduces the absorption of unrelated knowledge in each global iteration. Experimental results on MNIST, Digit-5, Office-10, and DomainNet show that our method outperforms multiple baselines, with an average test accuracy improvement of about 4.6\% and 10.4\% under feature and label non-IID distributions, respectively.

Related papers

• FedHPro: Federated Hyper-Prototype Learning via Gradient Matching [34.44446063708919]
  Federated Learning (FL) enables collaborative training of distributed clients while protecting privacy.<n>We introduce hyper-prototypes, defined by a set of learnable global class-wise prototypes to preserve semantic knowledge across clients.<n>We propose FedHPro, a Federated Hyper-Prototype Learning framework, to leverage hyper-prototypes to promote inter-class separability.
  arXiv  Detail & Related papers  (2026-05-13T13:01:17Z)
• Learning Clustering-based Prototypes for Compositional Zero-shot Learning [56.57299428499455]
  ClusPro is a robust clustering-based prototype mining framework for Compositional Zero-Shot Learning. It defines the conceptual boundaries of primitives through a set of diversified prototypes. ClusPro efficiently performs prototype clustering in a non-parametric fashion without the introduction of additional learnable parameters.
  arXiv  Detail & Related papers  (2025-02-10T14:20:01Z)
• Mitigating Domain Shift in Federated Learning via Intra- and Inter-Domain Prototypes [28.507105940018704]
  We introduce a novel federated prototype learning method, I$2$PFL, which incorporates $textbfI$ntra-domain and $textbfI$nter-domain. We propose feature alignment with MixUp-based augmented prototypes to capture the diversity within local domains and enhance the generalization of local features.
  arXiv  Detail & Related papers  (2025-01-15T02:17:38Z)
• An Enhanced Federated Prototype Learning Method under Domain Shift [36.73020712815063]
  Federated Learning (FL) allows collaborative machine learning training without sharing private data. Recent paper introduces variance-aware dual-level prototype clustering and uses a novel $alpha$-sparsity prototype loss. Evaluations on the Digit-5, Office-10, and DomainNet datasets show that our method performs better than existing approaches.
  arXiv  Detail & Related papers  (2024-09-27T09:28:27Z)
• LC-Protonets: Multi-label Few-shot learning for world music audio tagging [65.72891334156706]
  We introduce Label-Combination Prototypical Networks (LC-Protonets) to address the problem of multi-label few-shot classification. LC-Protonets generate one prototype per label combination, derived from the power set of labels present in the limited training items. Our method is applied to automatic audio tagging across diverse music datasets, covering various cultures and including both modern and traditional music.
  arXiv  Detail & Related papers  (2024-09-17T15:13:07Z)
• FedHide: Federated Learning by Hiding in the Neighbors [12.71494268219787]
  We propose a prototype-based federated learning method designed for embedding networks in classification or verification tasks. Our approach generates proxy class prototypes by linearly combining them with their nearest neighbors. This technique conceals the true class prototype while enabling clients to learn discriminative embedding networks.
  arXiv  Detail & Related papers  (2024-09-12T07:37:49Z)
• ProtoGMM: Multi-prototype Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentation [0.8213829427624407]
  Domain adaptive semantic segmentation aims to generate accurate and dense predictions for an unlabeled target domain. We propose the ProtoGMM model, which incorporates the GMM into contrastive losses to perform guided contrastive learning. To achieve increased intra-class semantic similarity, decreased inter-class similarity, and domain alignment between the source and target domains, we employ multi-prototype contrastive learning.
  arXiv  Detail & Related papers  (2024-06-27T14:50:50Z)
• Taming Cross-Domain Representation Variance in Federated Prototype Learning with Heterogeneous Data Domains [8.047147770476212]
  Federated learning (FL) allows collaborative machine learning training without sharing private data. While most FL methods assume identical data domains across clients, real-world scenarios often involve heterogeneous data domains. We introduce FedPLVM, which establishes variance-aware dual-level prototypes clustering and employs a novel $alpha$-sparsity prototype loss.
  arXiv  Detail & Related papers  (2024-03-14T02:36:16Z)
• Beyond the Prototype: Divide-and-conquer Proxies for Few-shot Segmentation [63.910211095033596]
  Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. We propose a simple yet versatile framework in the spirit of divide-and-conquer. Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
  arXiv  Detail & Related papers  (2022-04-21T06:21:14Z)
• BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation [74.93176783541332]
  Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data. To make up for the absence of source data, most existing methods introduced feature prototype based pseudo-labeling strategies. We propose a general class-Balanced Multicentric Dynamic prototype strategy for the SFDA task.
  arXiv  Detail & Related papers  (2022-04-06T13:23:02Z)
• Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
  We present a nonparametric semantic segmentation model based on non-learnable prototypes. Our framework yields compelling results over several datasets. We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
  arXiv  Detail & Related papers  (2022-03-28T21:15:32Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.