Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$-Means Clustering

• URL: http://arxiv.org/abs/2210.14664v1
• Date: Wed, 26 Oct 2022 12:28:53 GMT
• Title: Coresets for Vertical Federated Learning: Regularized Linear Regression and $K$-Means Clustering
• Abstract summary: We propose a unified framework by constructing coresets in a distributed fashion for communication-efficient VFL. We study two important learning tasks in the VFL setting: regularized linear regression and $k$-means clustering. We theoretically show that using coresets can drastically alleviate the communication complexity, while nearly maintain the solution quality.
• Score: 24.51727660771511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vertical federated learning (VFL), where data features are stored in multiple parties distributively, is an important area in machine learning. However, the communication complexity for VFL is typically very high. In this paper, we propose a unified framework by constructing coresets in a distributed fashion for communication-efficient VFL. We study two important learning tasks in the VFL setting: regularized linear regression and $k$-means clustering, and apply our coreset framework to both problems. We theoretically show that using coresets can drastically alleviate the communication complexity, while nearly maintain the solution quality. Numerical experiments are conducted to corroborate our theoretical findings.

Related papers

• UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity [82.84279796262818]
  We propose UniFed-VLM, a unified instruction tuning framework for Vision-Language Models (VLMs)<n>It consists of two key components: 1) Federated Compensated Subspace Aggregation (FedCSA), which performs subspace-aligned aggregation of parameter-efficient adapters.<n>We conduct experiments on multiple benchmark datasets, and the results show that UniFed-VLM achieves stronger average performance.
  arXiv  Detail & Related papers  (2026-08-16T04:01:04Z)
• Optimizing Server Placement for Vertical Federated Learning in Dynamic Edge/Fog Networks [49.91939413642465]
  We investigate the control and optimization of vertical federated learning (VFL) methods in which edge/fog devices contain separate data features.<n>In this setting, our proposed methodology, server controlled VFL in dynamic networks (SC-DN), first establishes the existence of a global first-order stationary point for every global round.<n>We show that our methodology demonstrates superior classification/regression performance and resource consumption savings than even greedy methodologies.
  arXiv  Detail & Related papers  (2026-05-10T23:30:18Z)
• Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly [42.31182713177944]
  This survey analyzes the real-world data distributions in potential Vertical Federated Learning (VFL) applications. We propose a novel data-oriented taxonomy of VFL algorithms based on real VFL data distributions. Based on these observations, we outline key research directions aimed at bridging the gap between current VFL research and real-world applications.
  arXiv  Detail & Related papers  (2025-02-12T07:03:32Z)
• Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning? [50.03434441234569]
  Federated Learning (FL) has gained significant popularity due to its effectiveness in training machine learning models across diverse sites without requiring direct data sharing. While various algorithms have shown that FL with local updates is a communication-efficient distributed learning framework, the generalization performance of FL with local updates has received comparatively less attention.
  arXiv  Detail & Related papers  (2024-09-05T19:00:18Z)
• SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead [75.87007729801304]
  SpaFL: a communication-efficient FL framework is proposed to optimize sparse model structures with low computational overhead. Experiments show that SpaFL improves accuracy while requiring much less communication and computing resources compared to sparse baselines.
  arXiv  Detail & Related papers  (2024-06-01T13:10:35Z)
• Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach [67.27031215756121]
  Federated learning (FL) is a machine learning paradigm that targets model training without gathering the local data over various data sources. Standard FL, which employs a single server, can only support a limited number of users, leading to degraded learning capability. In this work, we consider a multi-server FL framework, referred to as emphConfederated Learning (CFL) in order to accommodate a larger number of users.
  arXiv  Detail & Related papers  (2024-02-28T03:27:10Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• VertiBench: Advancing Feature Distribution Diversity in Vertical Federated Learning Benchmarks [31.08004805380727]
  This paper introduces two key factors affecting VFL performance - feature importance and feature correlation. We also introduce a real VFL dataset to address the deficit in image-image VFL scenarios.
  arXiv  Detail & Related papers  (2023-07-05T05:55:08Z)
• Federated Empirical Risk Minimization via Second-Order Method [18.548661105227488]
  We present an interior point method (IPM) to solve a general empirical risk minimization problem under the federated learning setting. We show that the communication complexity of each iteration of our IPM is $tildeO(d3/2)$, where $d$ is the dimension (i.e., number of features) of the dataset.
  arXiv  Detail & Related papers  (2023-05-27T14:23:14Z)
• GLASU: A Communication-Efficient Algorithm for Federated Learning with Vertically Distributed Graph Data [44.02629656473639]
  We propose a model splitting method that splits a backbone GNN across the clients and the server and a communication-efficient algorithm, GLASU, to train such a model. We offer a theoretical analysis and conduct extensive numerical experiments on real-world datasets, showing that the proposed algorithm effectively trains a GNN model, whose performance matches that of the backbone GNN when trained in a centralized manner.
  arXiv  Detail & Related papers  (2023-03-16T17:47:55Z)
• Achieving Model Fairness in Vertical Federated Learning [47.8598060954355]
  Vertical federated learning (VFL) enables multiple enterprises possessing non-overlapped features to strengthen their machine learning models without disclosing their private data and model parameters. VFL suffers from fairness issues, i.e., the learned model may be unfairly discriminatory over the group with sensitive attributes. We propose a fair VFL framework to tackle this problem.
  arXiv  Detail & Related papers  (2021-09-17T04:40:11Z)
• Multiple Kernel-Based Online Federated Learning [15.102346715690755]
  We propose a novel multiple kernel-based OFL (MK-OFL) as a non-trivial extension of OMKL. We theoretically prove that MK-OFL achieves the optimal sublinear regret bound when compared with the best function in hindsight.
  arXiv  Detail & Related papers  (2021-02-22T09:51:00Z)
• Hybrid Federated Learning: Algorithms and Implementation [61.0640216394349]
  Federated learning (FL) is a recently proposed distributed machine learning paradigm dealing with distributed and private data sets. We propose a new model-matching-based problem formulation for hybrid FL. We then propose an efficient algorithm that can collaboratively train the global and local models to deal with full and partial featured data.
  arXiv  Detail & Related papers  (2020-12-22T23:56:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.