Practical Vertical Federated Learning with Unsupervised Representation Learning

• URL: http://arxiv.org/abs/2208.10278v1
• Date: Sat, 13 Aug 2022 08:41:32 GMT
• Title: Practical Vertical Federated Learning with Unsupervised Representation Learning
• Authors: Zhaomin Wu, Qinbin Li, Bingsheng He
• Abstract summary: Federated learning enables multiple parties to collaboratively train a machine learning model without sharing their raw data. We propose a novel communication-efficient vertical federated learning algorithm named FedOnce, which requires only one-shot communication among parties. Our privacy-preserving technique significantly outperforms the state-of-the-art approaches under the same privacy budget.
• Score: 47.77625754666018
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As societal concerns on data privacy recently increase, we have witnessed data silos among multiple parties in various applications. Federated learning emerges as a new learning paradigm that enables multiple parties to collaboratively train a machine learning model without sharing their raw data. Vertical federated learning, where each party owns different features of the same set of samples and only a single party has the label, is an important and challenging topic in federated learning. Communication costs among different parties have been a major hurdle for practical vertical learning systems. In this paper, we propose a novel communication-efficient vertical federated learning algorithm named FedOnce, which requires only one-shot communication among parties. To improve model accuracy and provide privacy guarantee, FedOnce features unsupervised learning representations in the federated setting and privacy-preserving techniques based on moments accountant. The comprehensive experiments on 10 datasets demonstrate that FedOnce achieves close performance compared to state-of-the-art vertical federated learning algorithms with much lower communication costs. Meanwhile, our privacy-preserving technique significantly outperforms the state-of-the-art approaches under the same privacy budget.

Related papers

• Towards Privacy-Aware Causal Structure Learning in Federated Setting [27.5652887311069]
  We study a privacy-aware causal structure learning problem in the federated setting. We propose a novel Federated PC (FedPC) algorithm with two new strategies for preserving data privacy without centralizing data.
  arXiv  Detail & Related papers  (2022-11-13T14:54:42Z)
• Non-IID data and Continual Learning processes in Federated Learning: A long road ahead [58.720142291102135]
  Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. In this work, we formally classify data statistical heterogeneity and review the most remarkable learning strategies that are able to face it. At the same time, we introduce approaches from other machine learning frameworks, such as Continual Learning, that also deal with data heterogeneity and could be easily adapted to the Federated Learning settings.
  arXiv  Detail & Related papers  (2021-11-26T09:57:11Z)
• Federated Self-Supervised Contrastive Learning via Ensemble Similarity Distillation [42.05438626702343]
  This paper investigates the feasibility of learning good representation space with unlabeled client data in a federated scenario. We propose a novel self-supervised contrastive learning framework that supports architecture-agnostic local training and communication-efficient global aggregation.
  arXiv  Detail & Related papers  (2021-09-29T02:13:22Z)
• Federated Learning: A Signal Processing Perspective [144.63726413692876]
  Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local datasets, without explicitly exchanging the data. This article provides a unified systematic framework for federated learning in a manner that encapsulates and highlights the main challenges that are natural to treat using signal processing tools.
  arXiv  Detail & Related papers  (2021-03-31T15:14:39Z)
• Constrained Differentially Private Federated Learning for Low-bandwidth Devices [1.1470070927586016]
  This paper presents a novel privacy-preserving federated learning scheme. It provides theoretical privacy guarantees, as it is based on Differential Privacy. It reduces the upstream and downstream bandwidth by up to 99.9% compared to standard federated learning.
  arXiv  Detail & Related papers  (2021-02-27T22:25:06Z)
• Practical One-Shot Federated Learning for Cross-Silo Setting [114.76232507580067]
  One-shot federated learning is a promising approach to make federated learning applicable in cross-silo setting. We propose a practical one-shot federated learning algorithm named FedKT. By utilizing the knowledge transfer technique, FedKT can be applied to any classification models and can flexibly achieve differential privacy guarantees.
  arXiv  Detail & Related papers  (2020-10-02T14:09:10Z)
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning [151.47900584193025]
  We propose an asynchronous federated SGD (AFSGD-VP) algorithm and its SVRG and SAGA variants on the vertically partitioned data. To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
  arXiv  Detail & Related papers  (2020-08-14T08:08:15Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)
• Additively Homomorphical Encryption based Deep Neural Network for Asymmetrically Collaborative Machine Learning [12.689643742151516]
  preserving machine learning creates a constraint which limits further applications in finance sectors. We propose a new practical scheme of collaborative machine learning that one party owns data, but another party owns labels only. Our experiments on different datasets demonstrate not only stable training without accuracy, but also more than 100 times speedup.
  arXiv  Detail & Related papers  (2020-07-14T06:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.