FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training

• URL: http://arxiv.org/abs/2008.10838v3
• Date: Sat, 15 Jun 2024 14:55:58 GMT
• Title: FedCVT: Semi-supervised Vertical Federated Learning with Cross-view Training
• Authors: Yan Kang, Yang Liu, Xinle Liang,
• Abstract summary: Federated Cross-view Training (FedCVT) is a semi-supervised learning approach that improves the performance of a vertical federated learning model. FedCVT does not require parties to share their original data and model parameters, thus preserving data privacy.
• Score: 9.638604434238882
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning allows multiple parties to build machine learning models collaboratively without exposing data. In particular, vertical federated learning (VFL) enables participating parties to build a joint machine learning model based on distributed features of aligned samples. However, VFL requires all parties to share a sufficient amount of aligned samples. In reality, the set of aligned samples may be small, leaving the majority of the non-aligned data unused. In this article, we propose Federated Cross-view Training (FedCVT), a semi-supervised learning approach that improves the performance of the VFL model with limited aligned samples. More specifically, FedCVT estimates representations for missing features, predicts pseudo-labels for unlabeled samples to expand the training set, and trains three classifiers jointly based on different views of the expanded training set to improve the VFL model's performance. FedCVT does not require parties to share their original data and model parameters, thus preserving data privacy. We conduct experiments on NUS-WIDE, Vehicle, and CIFAR10 datasets. The experimental results demonstrate that FedCVT significantly outperforms vanilla VFL that only utilizes aligned samples. Finally, we perform ablation studies to investigate the contribution of each component of FedCVT to the performance of FedCVT. Code is available at https://github.com/yankang18/FedCVT

Related papers

• Vertical Federated Learning with Missing Features During Training and Inference [37.44022318612869]
  We propose a vertical federated learning method for efficient training and inference of neural network-based models. Our approach is simple yet effective, relying on the strategic sharing of parameters on task-sampling and inference. Numerical experiments show improved performance of LASER-VFL over the baselines.
  arXiv  Detail & Related papers  (2024-10-29T22:09:31Z)
• ConDistFL: Conditional Distillation for Federated Learning from Partially Annotated Data [5.210280120905009]
  "ConDistFL" is a framework to combine Federated Learning (FL) with knowledge distillation. We validate our framework on four distinct partially annotated abdominal CT datasets from the MSD and KiTS19 challenges. Our ablation study suggests that ConDistFL can perform well without frequent aggregation, reducing the communication cost of FL.
  arXiv  Detail & Related papers  (2023-08-08T06:07:49Z)
• Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models [76.410400238974]
  We propose TTA with feedback to rectify the model output and prevent the model from becoming blindly confident. A CLIP model is adopted as the reward model during TTA and provides feedback for the VLM. The proposed textitreinforcement learning with CLIP feedback(RLCF) framework is highly flexible and universal.
  arXiv  Detail & Related papers  (2023-05-29T11:03:59Z)
• FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings [51.09574369310246]
  Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models. We propose a novel cross-silo dataset suite focused on healthcare, FLamby, to bridge the gap between theory and practice of cross-silo FL. Our flexible and modular suite allows researchers to easily download datasets, reproduce results and re-use the different components for their research.
  arXiv  Detail & Related papers  (2022-10-10T12:17:30Z)
• Vertical Semi-Federated Learning for Efficient Online Advertising [50.18284051956359]
  Semi-VFL (Vertical Semi-Federated Learning) is proposed to achieve a practical industry application fashion for VFL. We build an inference-efficient single-party student model applicable to the whole sample space. New representation distillation methods are designed to extract cross-party feature correlations for both the overlapped and non-overlapped data.
  arXiv  Detail & Related papers  (2022-09-30T17:59:27Z)
• Multi-Model Federated Learning with Provable Guarantees [19.470024548995717]
  Federated Learning (FL) is a variant of distributed learning where devices collaborate to learn a model without sharing their data with the central server or each other. We refer to the process of multiple independent clients simultaneously in a federated setting using a common pool of clients as a multi-model edge FL.
  arXiv  Detail & Related papers  (2022-07-09T19:47:52Z)
• Fair and efficient contribution valuation for vertical federated learning [49.50442779626123]
  Federated learning is a popular technology for training machine learning models on distributed data sources without sharing data. The Shapley value (SV) is a provably fair contribution valuation metric originated from cooperative game theory. We propose a contribution valuation metric called vertical federated Shapley value (VerFedSV) based on SV.
  arXiv  Detail & Related papers  (2022-01-07T19:57:15Z)
• A Coupled Design of Exploiting Record Similarity for Practical Vertical Federated Learning [47.77625754666018]
  Federated learning is a learning paradigm to enable collaborative learning across different parties without revealing raw data. Most existing studies in vertical federated learning disregard the "record linkage" process. We design a novel coupled training paradigm, FedSim, that integrates one-to-many linkage into the training process.
  arXiv  Detail & Related papers  (2021-06-11T11:09:53Z)
• Multi-VFL: A Vertical Federated Learning System for Multiple Data and Label Owners [10.507522234243021]
  We propose a novel method, Multi Vertical Federated Learning (Multi-VFL), to train VFL models when there are multiple data and label owners. Our results show that using adaptive datasets for model aggregation fastens convergence and improves accuracy.
  arXiv  Detail & Related papers  (2021-06-10T03:00:57Z)
• FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data [12.815996963583641]
  Federated learning (FL) has been proposed to allow collaborative training of machine learning (ML) models among multiple parties. We propose FedV, a framework for secure gradient computation in vertical settings for several widely used ML models. We show a reduction of 10%-70% of training time and 80% to 90% in data transfer with respect to the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-03-05T19:59:29Z)
• Ensemble Distillation for Robust Model Fusion in Federated Learning [72.61259487233214]
  Federated Learning (FL) is a machine learning setting where many devices collaboratively train a machine learning model. In most of the current training schemes the central model is refined by averaging the parameters of the server model and the updated parameters from the client side. We propose ensemble distillation for model fusion, i.e. training the central classifier through unlabeled data on the outputs of the models from the clients.
  arXiv  Detail & Related papers  (2020-06-12T14:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.