Gradient-less Federated Gradient Boosting Trees with Learnable Learning Rates

• URL: http://arxiv.org/abs/2304.07537v3
• Date: Sun, 24 Mar 2024 09:42:39 GMT
• Title: Gradient-less Federated Gradient Boosting Trees with Learnable Learning Rates
• Authors: Chenyang Ma, Xinchi Qiu, Daniel J. Beutel, Nicholas D. Lane,
• Abstract summary: We develop an innovative framework for horizontal federated XGBoost. It simultaneously boosts privacy and communication efficiency by making the learning rates of the aggregated tree ensembles learnable. Our approach achieves performance comparable to the state-of-the-art method and effectively improves communication efficiency by lowering both communication rounds and communication overhead by factors ranging from 25x to 700x.
• Score: 17.68344542462656
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The privacy-sensitive nature of decentralized datasets and the robustness of eXtreme Gradient Boosting (XGBoost) on tabular data raise the needs to train XGBoost in the context of federated learning (FL). Existing works on federated XGBoost in the horizontal setting rely on the sharing of gradients, which induce per-node level communication frequency and serious privacy concerns. To alleviate these problems, we develop an innovative framework for horizontal federated XGBoost which does not depend on the sharing of gradients and simultaneously boosts privacy and communication efficiency by making the learning rates of the aggregated tree ensembles learnable. We conduct extensive evaluations on various classification and regression datasets, showing our approach achieves performance comparable to the state-of-the-art method and effectively improves communication efficiency by lowering both communication rounds and communication overhead by factors ranging from 25x to 700x. Project Page: https://flower.ai/blog/2023-04-19-xgboost-with-flower/

Related papers

• Bilateral Differentially Private Vertical Federated Boosted Decision Trees [10.952674399412405]
  Federated learning is a distributed machine learning paradigm that enables collaborative training across multiple parties while ensuring data privacy. In this paper, we propose a variant of vertical federated XGBoost with bilateral differential privacy guarantee: MaskedXGBoost. Our algorithm's superiority in both utility and efficiency has been validated on multiple datasets.
  arXiv  Detail & Related papers  (2025-04-30T15:37:44Z)
• Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning [67.49221252724229]
  E-health allows smart devices and medical institutions to collaboratively collect patients' data, which is trained by Artificial Intelligence (AI) technologies to help doctors make diagnosis. Applying federated learning in e-health faces many challenges. Medical data is both horizontally and vertically partitioned. A naive combination of HFL and VFL has limitations including low training efficiency, unsound convergence analysis, and lack of parameter tuning strategies.
  arXiv  Detail & Related papers  (2024-04-15T19:45:07Z)
• Adversarial Learning Data Augmentation for Graph Contrastive Learning in Recommendation [56.10351068286499]
  We propose Learnable Data Augmentation for Graph Contrastive Learning (LDA-GCL) Our methods include data augmentation learning and graph contrastive learning, which follow the InfoMin and InfoMax principles, respectively. In implementation, our methods optimize the adversarial loss function to learn data augmentation and effective representations of users and items.
  arXiv  Detail & Related papers  (2023-02-05T06:55:51Z)
• GraphLearner: Graph Node Clustering with Fully Learnable Augmentation [76.63963385662426]
  Contrastive deep graph clustering (CDGC) leverages the power of contrastive learning to group nodes into different clusters. We propose a Graph Node Clustering with Fully Learnable Augmentation, termed GraphLearner. It introduces learnable augmentors to generate high-quality and task-specific augmented samples for CDGC.
  arXiv  Detail & Related papers  (2022-12-07T10:19:39Z)
• Joint Data and Feature Augmentation for Self-Supervised Representation Learning on Point Clouds [4.723757543677507]
  We propose a fusion contrastive learning framework to combine data augmentations in Euclidean space and feature augmentations in feature space. We conduct extensive object classification experiments and object part segmentation experiments to validate the transferability of the proposed framework. Experimental results demonstrate that the proposed framework is effective to learn the point cloud representation in a self-supervised manner.
  arXiv  Detail & Related papers  (2022-11-02T14:58:03Z)
• FedGBF: An efficient vertical federated learning framework via gradient boosting and bagging [14.241194034190304]
  We propose a novel model in a vertically federated setting termed Federated Gradient Boosting Forest (FedGBF) FedGBF simultaneously integrates the boosting and bagging's preponderance by building the decision trees in parallel as a base learner for boosting. We also propose the Dynamic FedGBF, which dynamically changes each forest's parameters and thus reduces the complexity.
  arXiv  Detail & Related papers  (2022-04-03T03:03:34Z)
• GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
  Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
  arXiv  Detail & Related papers  (2022-03-24T02:58:36Z)
• Federated Knowledge Graphs Embedding [50.35484170815679]
  We propose a novel decentralized scalable learning framework, Federated Knowledge Graphs Embedding (FKGE) FKGE exploits adversarial generation between pairs of knowledge graphs to translate identical entities and relations of different domains into near embedding spaces. In order to protect the privacy of the training data, FKGE further implements a privacy-preserving neural network structure to guarantee no raw data leakage.
  arXiv  Detail & Related papers  (2021-05-17T05:30:41Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)
• Adaptive Histogram-Based Gradient Boosted Trees for Federated Learning [10.893840244877568]
  Federated Learning (FL) is an approach to collaboratively train a model across multiple parties without sharing data between parties or an aggregator. It is used both in the consumer domain to protect personal data as well as in enterprise settings, where dealing with data domicile regulation and the pragmatics of data silos are the main drivers. We propose a novel implementation of gradient boosting which utilizes a party adaptive histogram aggregation method, without the need for data encryption.
  arXiv  Detail & Related papers  (2020-12-11T23:01:35Z)
• FederBoost: Private Federated Learning for GBDT [45.903895659670674]
  Federated Learning (FL) has been an emerging trend in machine learning and artificial intelligence. We propose a framework named FederBoost for private federated learning of gradient boosting decision trees (GBDT)
  arXiv  Detail & Related papers  (2020-11-05T13:05:12Z)
• FedBoosting: Federated Learning with Gradient Protected Boosting for Text Recognition [7.988454173034258]
  Federated Learning (FL) framework allows learning a shared model collaboratively without data being centralized or shared among data owners. We show in this paper that the generalization ability of the joint model is poor on Non-Independent and Non-Identically Distributed (Non-IID) data. We propose a novel boosting algorithm for FL to address both the generalization and gradient leakage issues.
  arXiv  Detail & Related papers  (2020-07-14T18:47:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.