Learn How to Query from Unlabeled Data Streams in Federated Learning

• URL: http://arxiv.org/abs/2412.08138v2
• Date: Thu, 12 Dec 2024 01:47:20 GMT
• Title: Learn How to Query from Unlabeled Data Streams in Federated Learning
• Authors: Yuchang Sun, Xinran Li, Tao Lin, Jun Zhang,
• Abstract summary: Federated learning (FL) enables collaborative learning among decentralized clients while safeguarding the privacy of their local data. Existing studies on FL typically assume offline labeled data available at each client when the training starts. Given the expensive annotation cost, it is critical to identify a subset of informative samples for labeling on clients.
• Score: 8.963441227294235
• License:
• Abstract: Federated learning (FL) enables collaborative learning among decentralized clients while safeguarding the privacy of their local data. Existing studies on FL typically assume offline labeled data available at each client when the training starts. Nevertheless, the training data in practice often arrive at clients in a streaming fashion without ground-truth labels. Given the expensive annotation cost, it is critical to identify a subset of informative samples for labeling on clients. However, selecting samples locally while accommodating the global training objective presents a challenge unique to FL. In this work, we tackle this conundrum by framing the data querying process in FL as a collaborative decentralized decision-making problem and proposing an effective solution named LeaDQ, which leverages multi-agent reinforcement learning algorithms. In particular, under the implicit guidance from global information, LeaDQ effectively learns the local policies for distributed clients and steers them towards selecting samples that can enhance the global model's accuracy. Extensive simulations on image and text tasks show that LeaDQ advances the model performance in various FL scenarios, outperforming the benchmarking algorithms.

Related papers

• AFed: Algorithmic Fair Federated Learning [13.216737333440596]
  Federated Learning (FL) has gained significant attention as it facilitates collaborative machine learning among multiple clients without centralizing their data on a server. Traditional debiasing methods assume centralized access to sensitive information, rendering them impractical for the FL setting. This paper presents AFed, a framework for promoting group fairness in FL without access to client local data.
  arXiv  Detail & Related papers  (2025-01-06T03:05:49Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• FedSampling: A Better Sampling Strategy for Federated Learning [81.85411484302952]
  Federated learning (FL) is an important technique for learning models from decentralized data in a privacy-preserving way. Existing FL methods usually uniformly sample clients for local model learning in each round. We propose a novel data uniform sampling strategy for federated learning (FedSampling)
  arXiv  Detail & Related papers  (2023-06-25T13:38:51Z)
• When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning [41.51682329500003]
  We propose a novel learning rate adaptation mechanism to adjust the server learning rate for the aggregated gradient in each round. We make theoretical deductions to find a meaningful and robust indicator that is positively related to the optimal server learning rate.
  arXiv  Detail & Related papers  (2023-01-25T03:52:45Z)
• Knowledge-Aware Federated Active Learning with Non-IID Data [75.98707107158175]
  We propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget. The main challenge faced by federated active learning is the mismatch between the active sampling goal of the global model on the server and that of the local clients. We propose Knowledge-Aware Federated Active Learning (KAFAL), which consists of Knowledge-Specialized Active Sampling (KSAS) and Knowledge-Compensatory Federated Update (KCFU)
  arXiv  Detail & Related papers  (2022-11-24T13:08:43Z)
• FedNoiL: A Simple Two-Level Sampling Method for Federated Learning with Noisy Labels [49.47228898303909]
  Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. We develop a simple two-level sampling method "FedNoiL" that selects clients for more robust global aggregation on the server.
  arXiv  Detail & Related papers  (2022-05-20T12:06:39Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Distributed Unsupervised Visual Representation Learning with Fused Features [13.935997509072669]
  Federated learning (FL) enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. We propose a federated contrastive learning framework consisting of two approaches: feature fusion and neighborhood matching. It outperforms other methods by 11% on IID data and matches the performance of centralized learning.
  arXiv  Detail & Related papers  (2021-11-21T08:36:31Z)
• Semi-Supervised Federated Learning with non-IID Data: Algorithm and System Design [42.63120623012093]
  Federated Learning (FL) allows edge devices (or clients) to keep data locally while simultaneously training a shared global model. The distribution of the client's local training data is non-independent identically distributed (non-IID) We present a robust semi-supervised FL system design, where the system aims to solve the problem of data availability and non-IID in FL.
  arXiv  Detail & Related papers  (2021-10-26T03:41:48Z)
• Federated Noisy Client Learning [105.00756772827066]
  Federated learning (FL) collaboratively aggregates a shared global model depending on multiple local clients. Standard FL methods ignore the noisy client issue, which may harm the overall performance of the aggregated model. We propose Federated Noisy Client Learning (Fed-NCL), which is a plug-and-play algorithm and contains two main components.
  arXiv  Detail & Related papers  (2021-06-24T11:09:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.