Personalized Federated Learning via Active Sampling

• URL: http://arxiv.org/abs/2409.02064v2
• Date: Sun, 8 Sep 2024 08:29:34 GMT
• Title: Personalized Federated Learning via Active Sampling
• Authors: Alexander Jung, Yasmin SarcheshmehPour, Amirhossein Mohammadi,
• Abstract summary: This paper proposes a novel method for sequentially identifying similar (or relevant) data generators. Our method evaluates the relevance of a data generator by evaluating the effect of a gradient step using its local dataset. We extend this method to non-parametric models by a suitable generalization of the gradient step to update a hypothesis using the local dataset provided by a data generator.
• Score: 50.456464838807115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Consider a collection of data generators which could represent, e.g., humans equipped with a smart-phone or wearables. We want to train a personalized (or tailored) model for each data generator even if they provide only small local datasets. The available local datasets might fail to provide sufficient statistical power to train high-dimensional models (such as deep neural networks) effectively. One possible solution is to identify similar data generators and pool their local datasets to obtain a sufficiently large training set. This paper proposes a novel method for sequentially identifying similar (or relevant) data generators. Our method is similar in spirit to active sampling methods but does not require exchange of raw data. Indeed, our method evaluates the relevance of a data generator by evaluating the effect of a gradient step using its local dataset. This evaluation can be performed in a privacy-friendly fashion without sharing raw data. We extend this method to non-parametric models by a suitable generalization of the gradient step to update a hypothesis using the local dataset provided by a data generator.

Related papers

• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• Generative Dataset Distillation: Balancing Global Structure and Local Details [49.20086587208214]
  We propose a new dataset distillation method that considers balancing global structure and local details. Our method involves using a conditional generative adversarial network to generate the distilled dataset.
  arXiv  Detail & Related papers  (2024-04-26T23:46:10Z)
• Few-Shot Object Detection via Synthetic Features with Optimal Transport [28.072187044345107]
  We propose a novel approach in which we train a generator to generate synthetic data for novel classes. Our overarching goal is to train a generator that captures the data variations of the base dataset. We then transform the captured variations into novel classes by generating synthetic data with the trained generator.
  arXiv  Detail & Related papers  (2023-08-29T03:54:26Z)
• Improved Distribution Matching for Dataset Condensation [91.55972945798531]
  We propose a novel dataset condensation method based on distribution matching. Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
  arXiv  Detail & Related papers  (2023-07-19T04:07:33Z)
• Exploring Data Redundancy in Real-world Image Classification through Data Selection [20.389636181891515]
  Deep learning models often require large amounts of data for training, leading to increased costs. We present two data valuation metrics based on Synaptic Intelligence and gradient norms, respectively, to study redundancy in real-world image data. Online and offline data selection algorithms are then proposed via clustering and grouping based on the examined data values.
  arXiv  Detail & Related papers  (2023-06-25T03:31:05Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Achieving Representative Data via Convex Hull Feasibility Sampling Algorithms [35.29582673348303]
  Sampling biases in training data are a major source of algorithmic biases in machine learning systems. We present adaptive sampling methods to determine, with high confidence, whether it is possible to assemble a representative dataset from the given data sources.
  arXiv  Detail & Related papers  (2022-04-13T23:14:05Z)
• Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows [0.0]
  A strategy is proposed to select data points such that they uniformly span the phase-space of the data. An iterative method is used to accurately estimate the probability of the rare data points when only a small subset of the dataset is used to construct the probability map. The proposed framework is demonstrated as a viable pathway to enable data-efficient machine learning when abundant data is available.
  arXiv  Detail & Related papers  (2021-12-28T20:06:28Z)
• A Single Example Can Improve Zero-Shot Data Generation [7.237231992155901]
  Sub-tasks of intent classification require extensive and flexible datasets for experiments and evaluation. We propose to use text generation methods to gather datasets. We explore two approaches to generating task-oriented utterances.
  arXiv  Detail & Related papers  (2021-08-16T09:43:26Z)
• BREEDS: Benchmarks for Subpopulation Shift [98.90314444545204]
  We develop a methodology for assessing the robustness of models to subpopulation shift. We leverage the class structure underlying existing datasets to control the data subpopulations that comprise the training and test distributions. Applying this methodology to the ImageNet dataset, we create a suite of subpopulation shift benchmarks of varying granularity.
  arXiv  Detail & Related papers  (2020-08-11T17:04:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.