Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networks

• URL: http://arxiv.org/abs/2312.11230v1
• Date: Mon, 18 Dec 2023 14:30:05 GMT
• Title: Dirichlet-based Uncertainty Quantification for Personalized Federated Learning with Improved Posterior Networks
• Authors: Nikita Kotelevskii, Samuel Horv\'ath, Karthik Nandakumar, Martin Tak\'a\v{c}, Maxim Panov
• Abstract summary: This paper presents a new approach to federated learning that allows selecting a model from global and personalized ones. It is achieved through a careful modeling of predictive uncertainties that helps to detect local and global in- and out-of-distribution data. The comprehensive experimental evaluation on the popular real-world image datasets shows the superior performance of the model in the presence of out-of-distribution data.
• Score: 9.54563359677778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In modern federated learning, one of the main challenges is to account for inherent heterogeneity and the diverse nature of data distributions for different clients. This problem is often addressed by introducing personalization of the models towards the data distribution of the particular client. However, a personalized model might be unreliable when applied to the data that is not typical for this client. Eventually, it may perform worse for these data than the non-personalized global model trained in a federated way on the data from all the clients. This paper presents a new approach to federated learning that allows selecting a model from global and personalized ones that would perform better for a particular input point. It is achieved through a careful modeling of predictive uncertainties that helps to detect local and global in- and out-of-distribution data and use this information to select the model that is confident in a prediction. The comprehensive experimental evaluation on the popular real-world image datasets shows the superior performance of the model in the presence of out-of-distribution data while performing on par with state-of-the-art personalized federated learning algorithms in the standard scenarios.

Related papers

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Visual Prompt Based Personalized Federated Learning [83.04104655903846]
  We propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.
  arXiv  Detail & Related papers  (2023-03-15T15:02:15Z)
• A Closer Look at Personalization in Federated Image Classification [33.27317065917578]
  Federated Learning (FL) is developed to learn a single global model across the decentralized data. This paper shows that it is possible to achieve flexible personalization after the convergence of the global model. We propose RepPer, an independent two-stage personalized FL framework.
  arXiv  Detail & Related papers  (2022-04-22T06:32:18Z)
• Personalized Federated Learning through Local Memorization [10.925242558525683]
  Federated learning allows clients to collaboratively learn statistical models while keeping their data local. Recent personalized federated learning methods train a separate model for each client while still leveraging the knowledge available at other clients. We show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-17T19:40:07Z)
• New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning [5.500172106704342]
  In Federated Learning (FL), the clients learn a single global model (FedAvg) through a central aggregator. In this setting, the non-IID distribution of the data across clients restricts the global FL model from delivering good performance on the local data of each client. Personalized FL aims to address this problem by finding a personalized model for each client.
  arXiv  Detail & Related papers  (2021-07-28T05:30:17Z)
• Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
  Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
  arXiv  Detail & Related papers  (2021-04-27T18:23:54Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• Personalized Federated Learning with First Order Model Optimization [76.81546598985159]
  We propose an alternative to federated learning, where each client federates with other relevant clients to obtain a stronger model per client-specific objectives. We do not assume knowledge of underlying data distributions or client similarities, and allow each client to optimize for arbitrary target distributions of interest. Our method outperforms existing alternatives, while also enabling new features for personalized FL such as transfer outside of local data distributions.
  arXiv  Detail & Related papers  (2020-12-15T19:30:29Z)
• Specialized federated learning using a mixture of experts [0.6974741712647655]
  In federated learning, clients share a global model that has been trained on decentralized local client data. We propose an alternative method to learn a personalized model for each client in a federated setting. Our results show that the mixture of experts model is better suited as a personalized model for devices in these settings.
  arXiv  Detail & Related papers  (2020-10-05T14:43:57Z)
• Survey of Personalization Techniques for Federated Learning [0.08594140167290096]
  Federated learning enables machine learning models to learn from private decentralized data without compromising privacy. This paper highlights the need for personalization and surveys recent research on this topic.
  arXiv  Detail & Related papers  (2020-03-19T10:47:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.