Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data

• URL: http://arxiv.org/abs/2203.01752v1
• Date: Thu, 3 Mar 2022 14:58:29 GMT
• Title: Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data
• Authors: Yiu-ming Cheung, Juyong Jiang, Feng Yu, and Jian Lou
• Abstract summary: This paper will study the unsupervised federated learning under the vertically partitioned dataset setting. We propose the federated principal component analysis for vertically partitioned dataset (VFedPCA) method. We also take advantage of the nonlinear dimensionality reduction and propose the vertical federated advanced kernel principal component analysis (VFedAKPCA) method.
• Score: 35.72930187906397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite enormous research interest and rapid application of federated learning (FL) to various areas, existing studies mostly focus on supervised federated learning under the horizontally partitioned local dataset setting. This paper will study the unsupervised FL under the vertically partitioned dataset setting. Accordingly, we propose the federated principal component analysis for vertically partitioned dataset (VFedPCA) method, which reduces the dimensionality across the joint datasets over all the clients and extracts the principal component feature information for downstream data analysis. We further take advantage of the nonlinear dimensionality reduction and propose the vertical federated advanced kernel principal component analysis (VFedAKPCA) method, which can effectively and collaboratively model the nonlinear nature existing in many real datasets. In addition, we study two communication topologies. The first is a server-client topology where a semi-trusted server coordinates the federated training, while the second is the fully-decentralized topology which further eliminates the requirement of the server by allowing clients themselves to communicate with their neighbors. Extensive experiments conducted on five types of real-world datasets corroborate the efficacy of VFedPCA and VFedAKPCA under the vertically partitioned FL setting. Code is available at: https://github.com/juyongjiang/VFedPCA-VFedAKPCA

Related papers

• Vertical Federated Learning Hybrid Local Pre-training [4.31644387824845]
  We propose a novel VFL Hybrid Local Pre-training (VFLHLP) approach for Vertical Federated Learning (VFL) VFLHLP first pre-trains local networks on the local data of participating parties. Then it utilizes these pre-trained networks to adjust the sub-model for the labeled party or enhance representation learning for other parties during downstream federated learning on aligned data.
  arXiv  Detail & Related papers  (2024-05-20T08:57:39Z)
• FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation [24.78757412559944]
  Federated learning (FL) is an emerging paradigm for decentralized training of machine learning models on distributed clients. In this paper, we propose a generalized algorithm FedEmb, for modelling vertical and hybrid-based learning. The experimental results show that FedEmb is an effective method to tackle both split feature & subject space decentralized problems.
  arXiv  Detail & Related papers  (2023-11-30T16:01:51Z)
• An Efficient Virtual Data Generation Method for Reducing Communication in Federated Learning [34.39250699866746]
  A few classical schemes assume the server can extract the auxiliary information about training data of the participants from the local models to construct a central dummy dataset. The server uses the dummy dataset to finetune aggregated global model to achieve the target test accuracy in fewer communication rounds. In this paper, we summarize the above solutions into a data-based communication-efficient FL framework.
  arXiv  Detail & Related papers  (2023-06-21T08:07:07Z)
• Vertical Federated Learning over Cloud-RAN: Convergence Analysis and System Optimization [82.12796238714589]
  We propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation. We characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. We establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed.
  arXiv  Detail & Related papers  (2023-05-04T09:26:03Z)
• UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection [37.03465340777392]
  Dimensional reduction(DR) maps high-dimensional data into a lower dimensions latent space with minimized defined optimization objectives. FS focuses on selecting a critical subset of dimensions but risks destroying the data distribution (structure) FP combines all the input features into lower dimensions space, aiming to maintain the data structure; but lacks interpretability and sparsity. We develop a unified framework, Unified Dimensional Reduction Neural-network(UDRN), that integrates FS and FP in a compatible, end-to-end way.
  arXiv  Detail & Related papers  (2022-07-08T10:30:20Z)
• Fair and efficient contribution valuation for vertical federated learning [49.50442779626123]
  Federated learning is a popular technology for training machine learning models on distributed data sources without sharing data. The Shapley value (SV) is a provably fair contribution valuation metric originated from cooperative game theory. We propose a contribution valuation metric called vertical federated Shapley value (VerFedSV) based on SV.
  arXiv  Detail & Related papers  (2022-01-07T19:57:15Z)
• Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
  We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn. We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
  arXiv  Detail & Related papers  (2020-12-01T11:46:03Z)
• Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning [151.47900584193025]
  We propose an asynchronous federated SGD (AFSGD-VP) algorithm and its SVRG and SAGA variants on the vertically partitioned data. To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
  arXiv  Detail & Related papers  (2020-08-14T08:08:15Z)
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data [93.76907759950608]
  We propose a doubly kernel learning algorithm for vertically partitioned data. We show that FDSKL is significantly faster than state-of-the-art federated learning methods when dealing with kernels.
  arXiv  Detail & Related papers  (2020-08-14T05:46:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.