FederBoost: Private Federated Learning for GBDT

• URL: http://arxiv.org/abs/2011.02796v4
• Date: Thu, 27 Feb 2025 11:38:05 GMT
• Title: FederBoost: Private Federated Learning for GBDT
• Authors: Zhihua Tian, Rui Zhang, Xiaoyang Hou, Lingjuan Lyu, Tianyi Zhang, Jian Liu, Kui Ren,
• Abstract summary: Federated Learning (FL) has been an emerging trend in machine learning and artificial intelligence.<n>We propose a framework named FederBoost for private federated learning of gradient boosting decision trees (GBDT)
• Score: 45.903895659670674
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) has been an emerging trend in machine learning and artificial intelligence. It allows multiple participants to collaboratively train a better global model and offers a privacy-aware paradigm for model training since it does not require participants to release their original training data. However, existing FL solutions for vertically partitioned data or decision trees require heavy cryptographic operations. In this paper, we propose a framework named FederBoost for private federated learning of gradient boosting decision trees (GBDT). It supports running GBDT over both vertically and horizontally partitioned data. Vertical FederBoost does not require any cryptographic operation and horizontal FederBoost only requires lightweight secure aggregation. The key observation is that the whole training process of GBDT relies on the ordering of the data instead of the values. We fully implement FederBoost and evaluate its utility and efficiency through extensive experiments performed on three public datasets. Our experimental results show that both vertical and horizontal FederBoost achieve the same level of accuracy with centralized training where all data are collected in a central server, and they are 4-5 orders of magnitude faster than the state-of-the-art solutions for federated decision tree training; hence offering practical solutions for industrial applications.

Related papers

• Vertical Federated Learning Hybrid Local Pre-training [4.31644387824845]
  We propose a novel VFL Hybrid Local Pre-training (VFLHLP) approach for Vertical Federated Learning (VFL) VFLHLP first pre-trains local networks on the local data of participating parties. Then it utilizes these pre-trained networks to adjust the sub-model for the labeled party or enhance representation learning for other parties during downstream federated learning on aligned data.
  arXiv  Detail & Related papers  (2024-05-20T08:57:39Z)
• Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning [49.72857433721424]
  Vision Transformers (ViT) and Visual Prompt Tuning (VPT) achieve state-of-the-art performance with improved efficiency in various computer vision tasks. We present a novel algorithm, SGPT, that integrates Generalized FL (GFL) and Personalized FL (PFL) approaches by employing a unique combination of both shared and group-specific prompts.
  arXiv  Detail & Related papers  (2023-10-27T17:22:09Z)
• Gradient-less Federated Gradient Boosting Trees with Learnable Learning Rates [17.68344542462656]
  We develop an innovative framework for horizontal federated XGBoost. It simultaneously boosts privacy and communication efficiency by making the learning rates of the aggregated tree ensembles learnable. Our approach achieves performance comparable to the state-of-the-art method and effectively improves communication efficiency by lowering both communication rounds and communication overhead by factors ranging from 25x to 700x.
  arXiv  Detail & Related papers  (2023-04-15T11:48:18Z)
• FedDBL: Communication and Data Efficient Federated Deep-Broad Learning for Histopathological Tissue Classification [65.7405397206767]
  We propose Federated Deep-Broad Learning (FedDBL) to achieve superior classification performance with limited training samples and only one-round communication. FedDBL greatly outperforms the competitors with only one-round communication and limited training samples, while it even achieves comparable performance with the ones under multiple-round communications. Since no data or deep model sharing across different clients, the privacy issue is well-solved and the model security is guaranteed with no model inversion attack risk.
  arXiv  Detail & Related papers  (2023-02-24T14:27:41Z)
• FedABC: Targeting Fair Competition in Personalized Federated Learning [76.9646903596757]
  Federated learning aims to collaboratively train models without accessing their client's local private data. We propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the one-vs-all'' training strategy in each client to alleviate the unfair competition between classes.
  arXiv  Detail & Related papers  (2023-02-15T03:42:59Z)
• FedGBF: An efficient vertical federated learning framework via gradient boosting and bagging [14.241194034190304]
  We propose a novel model in a vertically federated setting termed Federated Gradient Boosting Forest (FedGBF) FedGBF simultaneously integrates the boosting and bagging's preponderance by building the decision trees in parallel as a base learner for boosting. We also propose the Dynamic FedGBF, which dynamically changes each forest's parameters and thus reduces the complexity.
  arXiv  Detail & Related papers  (2022-04-03T03:03:34Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• A Vertical Federated Learning Framework for Horizontally Partitioned Labels [12.433809611989155]
  Most existing vertical federated learning methods have a strong assumption that at least one party holds the complete set of labels of all data samples. Existing vertical federated learning methods can only utilize partial labels, which may lead to inadequate model update in end-to-end backpropagation. We propose a novel vertical federated learning framework named Cascade Vertical Federated Learning (CVFL) to fully utilize all horizontally partitioned labels to train neural networks with privacy-preservation.
  arXiv  Detail & Related papers  (2021-06-18T11:10:11Z)
• Adaptive Histogram-Based Gradient Boosted Trees for Federated Learning [10.893840244877568]
  Federated Learning (FL) is an approach to collaboratively train a model across multiple parties without sharing data between parties or an aggregator. It is used both in the consumer domain to protect personal data as well as in enterprise settings, where dealing with data domicile regulation and the pragmatics of data silos are the main drivers. We propose a novel implementation of gradient boosting which utilizes a party adaptive histogram aggregation method, without the need for data encryption.
  arXiv  Detail & Related papers  (2020-12-11T23:01:35Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.