Auto-weighted Robust Federated Learning with Corrupted Data Sources

• URL: http://arxiv.org/abs/2101.05880v1
• Date: Thu, 14 Jan 2021 21:54:55 GMT
• Title: Auto-weighted Robust Federated Learning with Corrupted Data Sources
• Authors: Shenghui Li, Edith Ngai, Fanghua Ye, and Thiemo Voigt
• Abstract summary: Federated learning provides a communication-efficient and privacy-preserving training process. Standard federated learning techniques that naively minimize an average loss function are vulnerable to data corruptions. We propose Auto-weighted Robust Federated Learning (arfl) to provide robustness against corrupted data sources.
• Score: 7.475348174281237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning provides a communication-efficient and privacy-preserving training process by enabling learning statistical models with massive participants while keeping their data in local clients. However, standard federated learning techniques that naively minimize an average loss function are vulnerable to data corruptions from outliers, systematic mislabeling, or even adversaries. In addition, it is often prohibited for service providers to verify the quality of data samples due to the increasing concern of user data privacy. In this paper, we address this challenge by proposing Auto-weighted Robust Federated Learning (arfl), a novel approach that jointly learns the global model and the weights of local updates to provide robustness against corrupted data sources. We prove a learning bound on the expected risk with respect to the predictor and the weights of clients, which guides the definition of the objective for robust federated learning. The weights are allocated by comparing the empirical loss of a client with the average loss of the best p clients (p-average), thus we can downweight the clients with significantly high losses, thereby lower their contributions to the global model. We show that this approach achieves robustness when the data of corrupted clients is distributed differently from benign ones. To optimize the objective function, we propose a communication-efficient algorithm based on the blockwise minimization paradigm. We conduct experiments on multiple benchmark datasets, including CIFAR-10, FEMNIST and Shakespeare, considering different deep neural network models. The results show that our solution is robust against different scenarios including label shuffling, label flipping and noisy features, and outperforms the state-of-the-art methods in most scenarios.

Related papers

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model [32.094290282897894]
  Federated learning aims to train a collective model from physically isolated clients while safeguarding the privacy of users' data. We propose a novel lightweight unsupervised federated learning approach that leverages unlabeled data on each client to perform lightweight model training and communication. Our proposed method greatly enhances model performance in comparison to CLIP's zero-shot predictions and even outperforms supervised federated learning benchmark methods.
  arXiv  Detail & Related papers  (2024-04-17T03:42:48Z)
• Reinforcement Learning as a Catalyst for Robust and Fair Federated Learning: Deciphering the Dynamics of Client Contributions [6.318638597489423]
  Reinforcement Federated Learning (RFL) is a novel framework that leverages deep reinforcement learning to adaptively optimize client contribution during aggregation. In terms of robustness, RFL outperforms state-of-the-art methods, while maintaining comparable levels of fairness.
  arXiv  Detail & Related papers  (2024-02-08T10:22:12Z)
• One-Shot Federated Learning with Classifier-Guided Diffusion Models [44.604485649167216]
  One-shot federated learning (OSFL) has gained attention in recent years due to its low communication cost. In this paper, we explore the novel opportunities that diffusion models bring to OSFL and propose FedCADO. FedCADO generates data that complies with clients' distributions and subsequently training the aggregated model on the server.
  arXiv  Detail & Related papers  (2023-11-15T11:11:25Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• Straggler-Resilient Personalized Federated Learning [55.54344312542944]
  Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
  arXiv  Detail & Related papers  (2022-06-05T01:14:46Z)
• Performance Weighting for Robust Federated Learning Against Corrupted Sources [1.76179873429447]
  Federated learning has emerged as a dominant computational paradigm for distributed machine learning. In real-world applications, a federated environment may consist of a mixture of benevolent and malicious clients. We show that the standard global aggregation scheme of local weights is inefficient in the presence of corrupted clients.
  arXiv  Detail & Related papers  (2022-05-02T20:01:44Z)
• Towards Fair Federated Learning with Zero-Shot Data Augmentation [123.37082242750866]
  Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. We propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server).
  arXiv  Detail & Related papers  (2021-04-27T18:23:54Z)
• Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
  Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
  arXiv  Detail & Related papers  (2020-12-20T14:34:36Z)
• CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning [2.2172881631608456]
  We introduce a new family of Federated Learning algorithms called CatFedAvg. It improves the communication efficiency but improves the quality of learning using a category coverage inNIST strategy. Our experiments show that an increase of 10% absolute points accuracy using the M dataset with 70% absolute points lower network transfer over FedAvg.
  arXiv  Detail & Related papers  (2020-11-14T06:52:02Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.