Three Approaches for Personalization with Applications to Federated Learning

• URL: http://arxiv.org/abs/2002.10619v2
• Date: Sun, 19 Jul 2020 21:02:14 GMT
• Title: Three Approaches for Personalization with Applications to Federated Learning
• Authors: Yishay Mansour and Mehryar Mohri and Jae Ro and Ananda Theertha Suresh
• Abstract summary: We present a systematic learning-theoretic study of personalization. We provide learning-theoretic guarantees and efficient algorithms for which we also demonstrate the performance. All of our algorithms are model-agnostic and work for any hypothesis class.
• Score: 68.19709953755238
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The standard objective in machine learning is to train a single model for all users. However, in many learning scenarios, such as cloud computing and federated learning, it is possible to learn a personalized model per user. In this work, we present a systematic learning-theoretic study of personalization. We propose and analyze three approaches: user clustering, data interpolation, and model interpolation. For all three approaches, we provide learning-theoretic guarantees and efficient algorithms for which we also demonstrate the performance empirically. All of our algorithms are model-agnostic and work for any hypothesis class.

Related papers

• Learn What You Need in Personalized Federated Learning [53.83081622573734]
  $textitLearn2pFed$ is a novel algorithm-unrolling-based personalized federated learning framework. We show that $textitLearn2pFed$ significantly outperforms previous personalized federated learning methods.
  arXiv  Detail & Related papers  (2024-01-16T12:45:15Z)
• FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization [1.911678487931003]
  Federated learning seeks to integrate the training learning models from multiple users, each user having their own data set, in a way that is sensitive to data privacy and to communication loss constraints. In this paper, we propose a novel solution to a global, clustered problem of federated learning that is inspired by ideas in consensus-based optimization (CBO) Our new CBO-type method is based on a system of interacting particles that is oblivious to group.
  arXiv  Detail & Related papers  (2023-05-04T15:02:09Z)
• Algorithmic Collective Action in Machine Learning [35.91866986642348]
  We study algorithmic collective action on digital platforms that deploy machine learning algorithms. We propose a simple theoretical model of a collective interacting with a firm's learning algorithm. We conduct systematic experiments on a skill classification task involving tens of thousands of resumes from a gig platform for freelancers.
  arXiv  Detail & Related papers  (2023-02-08T18:55:49Z)
• A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy [10.27527187262914]
  We develop a generative framework that could unify several different algorithms as well as suggest new algorithms. We then use our generative framework for learning, which unifies several known personalized FL algorithms and also suggests new ones. We also develop privacy for personalized learning methods with guarantees for user-level privacy and composition.
  arXiv  Detail & Related papers  (2022-07-05T02:18:44Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Straggler-Resilient Personalized Federated Learning [55.54344312542944]
  Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
  arXiv  Detail & Related papers  (2022-06-05T01:14:46Z)
• A Personalized Federated Learning Algorithm: an Application in Anomaly Detection [0.6700873164609007]
  Federated Learning (FL) has recently emerged as a promising method to overcome data privacy and transmission issues. In FL, datasets collected from different devices or sensors are used to train local models (clients) each of which shares its learning with a centralized model (server) This paper proposes a novel Personalized FedAvg (PC-FedAvg) which aims to control weights communication and aggregation augmented with a tailored learning algorithm to personalize the resulting models at each client.
  arXiv  Detail & Related papers  (2021-11-04T04:57:11Z)
• Partner-Assisted Learning for Few-Shot Image Classification [54.66864961784989]
  Few-shot Learning has been studied to mimic human visual capabilities and learn effective models without the need of exhaustive human annotation. In this paper, we focus on the design of training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples. We propose a two-stage training scheme, which first trains a partner encoder to model pair-wise similarities and extract features serving as soft-anchors, and then trains a main encoder by aligning its outputs with soft-anchors while attempting to maximize classification performance.
  arXiv  Detail & Related papers  (2021-09-15T22:46:19Z)
• Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation [71.76008290101214]
  DistillFlow is a knowledge distillation approach to learning optical flow. It achieves state-of-the-art unsupervised learning performance on both KITTI and Sintel datasets. Our models ranked 1st among all monocular methods on the KITTI 2015 benchmark, and outperform all published methods on the Sintel Final benchmark.
  arXiv  Detail & Related papers  (2021-06-08T09:13:34Z)
• Small-Group Learning, with Application to Neural Architecture Search [17.86826990290058]
  In human learning, a small group of students work together towards the same learning objective, where they express their understanding of a topic to their peers, compare their ideas, and help each other to trouble-shoot problems. In this paper, we aim to investigate whether this human learning method can be borrowed to train better machine learning models, by developing a novel ML framework -- small-group learning (SGL) SGL is formulated as a multi-level optimization framework consisting of three learning stages: each learner trains a model independently and uses this model to perform pseudo-labeling; each learner trains another model using datasets pseudo-
  arXiv  Detail & Related papers  (2020-12-23T05:56:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.