Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations

• URL: http://arxiv.org/abs/2407.02073v1
• Date: Tue, 2 Jul 2024 09:05:43 GMT
• Title: Contribution Evaluation of Heterogeneous Participants in Federated Learning via Prototypical Representations
• Authors: Qi Guo, Minghao Yao, Zhen Tian, Saiyu Qi, Yong Qi, Yun Lin, Jin Song Dong,
• Abstract summary: contribution evaluation in federated learning (FL) has become a pivotal research area due to its applicability across various domains. Existing contribution evaluation methods, which primarily rely on data volume, model similarity, and auxiliary test datasets, have shown success in diverse scenarios. This paper explores contribution evaluation in FL from an entirely new perspective of representation.
• Score: 18.73128175231337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contribution evaluation in federated learning (FL) has become a pivotal research area due to its applicability across various domains, such as detecting low-quality datasets, enhancing model robustness, and designing incentive mechanisms. Existing contribution evaluation methods, which primarily rely on data volume, model similarity, and auxiliary test datasets, have shown success in diverse scenarios. However, their effectiveness often diminishes due to the heterogeneity of data distributions, presenting a significant challenge to their applicability. In response, this paper explores contribution evaluation in FL from an entirely new perspective of representation. In this work, we propose a new method for the contribution evaluation of heterogeneous participants in federated learning (FLCE), which introduces a novel indicator \emph{class contribution momentum} to conduct refined contribution evaluation. Our core idea is the construction and application of the class contribution momentum indicator from individual, relative, and holistic perspectives, thereby achieving an effective and efficient contribution evaluation of heterogeneous participants without relying on an auxiliary test dataset. Extensive experimental results demonstrate the superiority of our method in terms of fidelity, effectiveness, efficiency, and heterogeneity across various scenarios.

Related papers

• DPVS-Shapley:Faster and Universal Contribution Evaluation Component in Federated Learning [1.740992908651449]
  We introduce a component called Dynamic Pruning Validation Set Shapley (DPVS-Shapley) This method accelerates the contribution assessment process by dynamically pruning the original dataset without compromising the evaluation's accuracy.
  arXiv  Detail & Related papers  (2024-10-19T13:01:44Z)
• CoAst: Validation-Free Contribution Assessment for Federated Learning based on Cross-Round Valuation [10.579048525756797]
  CoAst is a practical method to assess the contribution without access to any validation data. CoAst has comparable assessment reliability to existing validation-based methods and outperforms existing validation-free methods.
  arXiv  Detail & Related papers  (2024-09-04T07:46:28Z)
• Multi-Agent Reinforcement Learning from Human Feedback: Data Coverage and Algorithmic Techniques [65.55451717632317]
  We study Multi-Agent Reinforcement Learning from Human Feedback (MARLHF), exploring both theoretical foundations and empirical validations. We define the task as identifying Nash equilibrium from a preference-only offline dataset in general-sum games. Our findings underscore the multifaceted approach required for MARLHF, paving the way for effective preference-based multi-agent systems.
  arXiv  Detail & Related papers  (2024-09-01T13:14:41Z)
• Redefining Contributions: Shapley-Driven Federated Learning [3.9539878659683363]
  Federated learning (FL) has emerged as a pivotal approach in machine learning. It is challenging to ensure global model convergence when participants do not contribute equally and/or honestly. This paper proposes a novel contribution assessment method called ShapFed for fine-grained evaluation of participant contributions in FL.
  arXiv  Detail & Related papers  (2024-06-01T22:40:31Z)
• A Large-Scale Empirical Study on Improving the Fairness of Image Classification Models [22.522156479335706]
  This paper conducts the first large-scale empirical study to compare the performance of existing state-of-the-art fairness improving techniques. Our findings reveal substantial variations in the performance of each method across different datasets and sensitive attributes. Different fairness evaluation metrics, due to their distinct focuses, yield significantly different assessment results.
  arXiv  Detail & Related papers  (2024-01-08T06:53:33Z)
• Evaluating and Incentivizing Diverse Data Contributions in Collaborative Learning [89.21177894013225]
  For a federated learning model to perform well, it is crucial to have a diverse and representative dataset. We show that the statistical criterion used to quantify the diversity of the data, as well as the choice of the federated learning algorithm used, has a significant effect on the resulting equilibrium. We leverage this to design simple optimal federated learning mechanisms that encourage data collectors to contribute data representative of the global population.
  arXiv  Detail & Related papers  (2023-06-08T23:38:25Z)
• Evaluation of Self-taught Learning-based Representations for Facial Emotion Recognition [62.30451764345482]
  This work describes different strategies to generate unsupervised representations obtained through the concept of self-taught learning for facial emotion recognition. The idea is to create complementary representations promoting diversity by varying the autoencoders' initialization, architecture, and training data. Experimental results on Jaffe and Cohn-Kanade datasets using a leave-one-subject-out protocol show that FER methods based on the proposed diverse representations compare favorably against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-04-26T22:48:15Z)
• CAFE: Learning to Condense Dataset by Aligning Features [72.99394941348757]
  We propose a novel scheme to Condense dataset by Aligning FEatures (CAFE) At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art.
  arXiv  Detail & Related papers  (2022-03-03T05:58:49Z)
• Self-supervised Co-training for Video Representation Learning [103.69904379356413]
  We investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation training. We propose a novel self-supervised co-training scheme to improve the popular infoNCE loss. We evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval.
  arXiv  Detail & Related papers  (2020-10-19T17:59:01Z)
• Spectrum-Guided Adversarial Disparity Learning [52.293230153385124]
  We propose a novel end-to-end knowledge directed adversarial learning framework. It portrays the class-conditioned intraclass disparity using two competitive encoding distributions and learns the purified latent codes by denoising learned disparity. The experiments on four HAR benchmark datasets demonstrate the robustness and generalization of our proposed methods over a set of state-of-the-art.
  arXiv  Detail & Related papers  (2020-07-14T05:46:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.