Personalization Improves Privacy-Accuracy Tradeoffs in Federated Optimization

• URL: http://arxiv.org/abs/2202.05318v1
• Date: Thu, 10 Feb 2022 20:44:44 GMT
• Title: Personalization Improves Privacy-Accuracy Tradeoffs in Federated Optimization
• Authors: Alberto Bietti, Chen-Yu Wei, Miroslav Dudik, John Langford, Zhiwei Steven Wu
• Abstract summary: We show that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.
• Score: 57.98426940386627
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale machine learning systems often involve data distributed across a collection of users. Federated optimization algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We show that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.

Related papers

• Personalized Federated Learning for Cross-view Geo-localization [49.40531019551957]
  We propose a methodology combining Federated Learning (FL) with Cross-view Image Geo-localization (CVGL) techniques. Our method implements a coarse-to-fine approach, where clients share only the coarse feature extractors while keeping fine-grained features specific to local environments. Results demonstrate that our federated CVGL method achieves performance close to centralized training while maintaining data privacy.
  arXiv  Detail & Related papers  (2024-11-07T13:25:52Z)
• Can Public Large Language Models Help Private Cross-device Federated Learning? [58.05449579773249]
  We study (differentially) private federated learning (FL) of language models. Public data has been used to improve privacy-utility trade-offs for both large and small language models. We propose a novel distribution matching algorithm with theoretical grounding to sample public data close to private data distribution.
  arXiv  Detail & Related papers  (2023-05-20T07:55:58Z)
• Collaborative Chinese Text Recognition with Personalized Federated Learning [61.34060587461462]
  In Chinese text recognition, it is often necessary for one organization to collect a large amount of data from similar organizations. Due to the natural presence of private information in text data, such as addresses and phone numbers, different organizations are unwilling to share private data. We introduce personalized federated learning (pFL) into the Chinese text recognition task and propose the pFedCR algorithm.
  arXiv  Detail & Related papers  (2023-05-09T16:51:00Z)
• FedPDC:Federated Learning for Public Dataset Correction [1.5533842336139065]
  Federated learning has lower classification accuracy than traditional machine learning in Non-IID scenarios. New algorithm FedPDC is proposed to optimize the aggregation mode of local models and the loss function of local training. In many benchmark experiments, FedPDC can effectively improve the accuracy of the global model in the case of extremely unbalanced data distribution.
  arXiv  Detail & Related papers  (2023-02-24T08:09:23Z)
• Personalized Federated Learning through Local Memorization [10.925242558525683]
  Federated learning allows clients to collaboratively learn statistical models while keeping their data local. Recent personalized federated learning methods train a separate model for each client while still leveraging the knowledge available at other clients. We show on a suite of federated datasets that this approach achieves significantly higher accuracy and fairness than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-17T19:40:07Z)
• Subspace Learning for Personalized Federated Optimization [7.475183117508927]
  We propose a method to address the problem of personalized learning in AI systems. We show that our method achieves consistent gains both in personalized and unseen client evaluation settings.
  arXiv  Detail & Related papers  (2021-09-16T00:03:23Z)
• Specialized federated learning using a mixture of experts [0.6974741712647655]
  In federated learning, clients share a global model that has been trained on decentralized local client data. We propose an alternative method to learn a personalized model for each client in a federated setting. Our results show that the mixture of experts model is better suited as a personalized model for devices in these settings.
  arXiv  Detail & Related papers  (2020-10-05T14:43:57Z)
• Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification [69.29602103582782]
  Deep learning has been successful for many computer vision tasks due to the availability of shared and centralised large-scale training data. However, increasing awareness of privacy concerns poses new challenges to deep learning, especially for person re-identification (Re-ID) We propose a novel paradigm called Federated Person Re-Identification (FedReID) to construct a generalisable global model (a central server) by simultaneously learning with multiple privacy-preserved local models (local clients) This client-server collaborative learning process is iteratively performed under privacy control, enabling FedReID to realise decentralised learning without sharing distributed data nor collecting any
  arXiv  Detail & Related papers  (2020-06-07T13:32:33Z)
• Multi-Center Federated Learning [62.57229809407692]
  This paper proposes a novel multi-center aggregation mechanism for federated learning. It learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.
  arXiv  Detail & Related papers  (2020-05-03T09:14:31Z)
• Survey of Personalization Techniques for Federated Learning [0.08594140167290096]
  Federated learning enables machine learning models to learn from private decentralized data without compromising privacy. This paper highlights the need for personalization and surveys recent research on this topic.
  arXiv  Detail & Related papers  (2020-03-19T10:47:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.