Applied Federated Learning: Architectural Design for Robust and Efficient Learning in Privacy Aware Settings

• URL: http://arxiv.org/abs/2206.00807v1
• Date: Thu, 2 Jun 2022 00:30:04 GMT
• Title: Applied Federated Learning: Architectural Design for Robust and Efficient Learning in Privacy Aware Settings
• Authors: Branislav Stojkovic, Jonathan Woodbridge, Zhihan Fang, Jerry Cai, Andrey Petrov, Sathya Iyer, Daoyu Huang, Patrick Yau, Arvind Sastha Kumar, Hitesh Jawa, Anamita Guha
• Abstract summary: The classical machine learning paradigm requires the aggregation of user data in a central location. Centralization of data poses risks, including a heightened risk of internal and external security incidents. Federated learning with differential privacy is designed to avoid the server-side centralization pitfall.
• Score: 0.8454446648908585
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The classical machine learning paradigm requires the aggregation of user data in a central location where machine learning practitioners can preprocess data, calculate features, tune models and evaluate performance. The advantage of this approach includes leveraging high performance hardware (such as GPUs) and the ability of machine learning practitioners to do in depth data analysis to improve model performance. However, these advantages may come at a cost to data privacy. User data is collected, aggregated, and stored on centralized servers for model development. Centralization of data poses risks, including a heightened risk of internal and external security incidents as well as accidental data misuse. Federated learning with differential privacy is designed to avoid the server-side centralization pitfall by bringing the ML learning step to users' devices. Learning is done in a federated manner where each mobile device runs a training loop on a local copy of a model. Updates from on-device models are sent to the server via encrypted communication and through differential privacy to improve the global model. In this paradigm, users' personal data remains on their devices. Surprisingly, model training in this manner comes at a fairly minimal degradation in model performance. However, federated learning comes with many other challenges due to its distributed nature, heterogeneous compute environments and lack of data visibility. This paper explores those challenges and outlines an architectural design solution we are exploring and testing to productionize federated learning at Meta scale.

Related papers

• Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation [46.86767774669831]
  We propose a more effective and efficient federated unlearning scheme based on the concept of model explanation. We select the most influential channels within an already-trained model for the data that need to be unlearned.
  arXiv  Detail & Related papers  (2024-06-18T11:43:20Z)
• FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning [16.86560475992975]
  We introduce Attention-based Machine Unlearning using Federated Reinforcement Learning (FRAMU) FRAMU incorporates adaptive learning mechanisms, privacy preservation techniques, and optimization strategies. Our experiments, conducted on both single-modality and multi-modality datasets, revealed that FRAMU significantly outperformed baseline models.
  arXiv  Detail & Related papers  (2023-09-19T03:13:17Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Privacy-Preserving Machine Learning for Collaborative Data Sharing via Auto-encoder Latent Space Embeddings [57.45332961252628]
  Privacy-preserving machine learning in data-sharing processes is an ever-critical task. This paper presents an innovative framework that uses Representation Learning via autoencoders to generate privacy-preserving embedded data.
  arXiv  Detail & Related papers  (2022-11-10T17:36:58Z)
• Federated Learning and Meta Learning: Approaches, Applications, and Directions [94.68423258028285]
  In this tutorial, we present a comprehensive review of FL, meta learning, and federated meta learning (FedMeta) Unlike other tutorial papers, our objective is to explore how FL, meta learning, and FedMeta methodologies can be designed, optimized, and evolved, and their applications over wireless networks.
  arXiv  Detail & Related papers  (2022-10-24T10:59:29Z)
• Federated Split GANs [12.007429155505767]
  We propose an alternative approach to train ML models in user's devices themselves. We focus on GANs (generative adversarial networks) and leverage their inherent privacy-preserving attribute. Our system preserves data privacy, keeps a short training time, and yields same accuracy of model training in unconstrained devices.
  arXiv  Detail & Related papers  (2022-07-04T23:53:47Z)
• Comparative assessment of federated and centralized machine learning [0.0]
  Federated Learning (FL) is a privacy preserving machine learning scheme, where training happens with data federated across devices. In this paper, we discuss the various factors that affect the federated learning training, because of the non-IID distributed nature of the data. We show that federated learning does have an advantage in cost when the model sizes to be trained are not reasonably large.
  arXiv  Detail & Related papers  (2022-02-03T11:20:47Z)
• A Personalized Federated Learning Algorithm: an Application in Anomaly Detection [0.6700873164609007]
  Federated Learning (FL) has recently emerged as a promising method to overcome data privacy and transmission issues. In FL, datasets collected from different devices or sensors are used to train local models (clients) each of which shares its learning with a centralized model (server) This paper proposes a novel Personalized FedAvg (PC-FedAvg) which aims to control weights communication and aggregation augmented with a tailored learning algorithm to personalize the resulting models at each client.
  arXiv  Detail & Related papers  (2021-11-04T04:57:11Z)
• Decentralized Federated Learning Preserves Model and Data Privacy [77.454688257702]
  We propose a fully decentralized approach, which allows to share knowledge between trained models. Students are trained on the output of their teachers via synthetically generated input data. The results show that an untrained student model, trained on the teachers output reaches comparable F1-scores as the teacher.
  arXiv  Detail & Related papers  (2021-02-01T14:38:54Z)
• Information-Theoretic Bounds on the Generalization Error and Privacy Leakage in Federated Learning [96.38757904624208]
  Machine learning algorithms on mobile networks can be characterized into three different categories. The main objective of this work is to provide an information-theoretic framework for all of the aforementioned learning paradigms.
  arXiv  Detail & Related papers  (2020-05-05T21:23:45Z)
• Think Locally, Act Globally: Federated Learning with Local and Global Representations [92.68484710504666]
  Federated learning is a method of training models on private data distributed over multiple devices. We propose a new federated learning algorithm that jointly learns compact local representations on each device. We also evaluate on the task of personalized mood prediction from real-world mobile data where privacy is key.
  arXiv  Detail & Related papers  (2020-01-06T12:40:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.