Bilateral Differentially Private Vertical Federated Boosted Decision Trees

• URL: http://arxiv.org/abs/2504.21739v1
• Date: Wed, 30 Apr 2025 15:37:44 GMT
• Title: Bilateral Differentially Private Vertical Federated Boosted Decision Trees
• Authors: Bokang Zhang, Zhikun Zhang, Haodong Jiang, Yang Liu, Lihao Zheng, Yuxiao Zhou, Shuaiting Huang, Junfeng Wu,
• Abstract summary: Federated learning is a distributed machine learning paradigm that enables collaborative training across multiple parties while ensuring data privacy.<n>In this paper, we propose a variant of vertical federated XGBoost with bilateral differential privacy guarantee: MaskedXGBoost.<n>Our algorithm's superiority in both utility and efficiency has been validated on multiple datasets.
• Score: 10.952674399412405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning is a distributed machine learning paradigm that enables collaborative training across multiple parties while ensuring data privacy. Gradient Boosting Decision Trees (GBDT), such as XGBoost, have gained popularity due to their high performance and strong interpretability. Therefore, there has been a growing interest in adapting XGBoost for use in federated settings via cryptographic techniques. However, it should be noted that these approaches may not always provide rigorous theoretical privacy guarantees, and they often come with a high computational cost in terms of time and space requirements. In this paper, we propose a variant of vertical federated XGBoost with bilateral differential privacy guarantee: MaskedXGBoost. We build well-calibrated noise to perturb the intermediate information to protect privacy. The noise is structured with part of its ingredients in the null space of the arithmetical operation for splitting score evaluation in XGBoost, helping us achieve consistently better utility than other perturbation methods and relatively lower overhead than encryption-based techniques. We provide theoretical utility analysis and empirically verify privacy preservation. Compared with other algorithms, our algorithm's superiority in both utility and efficiency has been validated on multiple datasets.

Related papers

• Secure Federated XGBoost with CUDA-accelerated Homomorphic Encryption via NVIDIA FLARE [6.053716038605071]
  Federated learning (FL) enables collaborative model training across decentralized datasets.<n> NVIDIA FLARE's Federated XGBoost extends the popular XGBoost algorithm to both vertical and horizontal federated settings.<n>Initial implementation assumed mutual trust over the sharing of intermediate statistics.<n>We introduce "Secure Federated XGBoost", an efficient solution to mitigate these risks.
  arXiv  Detail & Related papers  (2025-04-04T20:08:24Z)
• Linear-Time User-Level DP-SCO via Robust Statistics [55.350093142673316]
  User-level differentially private convex optimization (DP-SCO) has garnered significant attention due to the importance of safeguarding user privacy in machine learning applications.<n>Current methods, such as those based on differentially private gradient descent (DP-SGD), often struggle with high noise accumulation and suboptimal utility.<n>We introduce a novel linear-time algorithm that leverages robust statistics, specifically the median and trimmed mean, to overcome these challenges.
  arXiv  Detail & Related papers  (2025-02-13T02:05:45Z)
• Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy [10.396575601912673]
  We introduce a federated learning algorithm called Differentially Private Federated Cubic Regularized Newton (DP-FCRN)<n>By leveraging second-order techniques, our algorithm achieves lower iteration complexity compared to first-order methods.<n>We also incorporate noise perturbation during local computations to ensure privacy.
  arXiv  Detail & Related papers  (2024-08-08T08:48:54Z)
• TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data [50.797729676285876]
  We propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm.
  arXiv  Detail & Related papers  (2024-02-16T16:41:14Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• On Differential Privacy and Adaptive Data Analysis with Bounded Space [76.10334958368618]
  We study the space complexity of the two related fields of differential privacy and adaptive data analysis. We show that there exists a problem P that requires exponentially more space to be solved efficiently with differential privacy. The line of work on adaptive data analysis focuses on understanding the number of samples needed for answering a sequence of adaptive queries.
  arXiv  Detail & Related papers  (2023-02-11T14:45:31Z)
• Federated Boosted Decision Trees with Differential Privacy [24.66980518231163]
  We propose a general framework that captures and extends existing approaches for differentially private decision trees. We show that with a careful choice of techniques it is possible to achieve very high utility while maintaining strong levels of privacy.
  arXiv  Detail & Related papers  (2022-10-06T13:28:29Z)
• Private Boosted Decision Trees via Smooth Re-Weighting [2.099922236065961]
  Differential Privacy is the appropriate mathematical framework for formal guarantees of privacy. We propose and test a practical algorithm for boosting decision trees that guarantees differential privacy.
  arXiv  Detail & Related papers  (2022-01-29T20:08:52Z)
• Differentially Private Federated Bayesian Optimization with Distributed Exploration [48.9049546219643]
  We introduce differential privacy (DP) into the training of deep neural networks through a general framework for adding DP to iterative algorithms. We show that DP-FTS-DE achieves high utility (competitive performance) with a strong privacy guarantee. We also use real-world experiments to show that DP-FTS-DE induces a trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2021-10-27T04:11:06Z)
• FedXGBoost: Privacy-Preserving XGBoost for Federated Learning [10.304484601250948]
  Federated learning is the distributed machine learning framework that enables collaborative training across multiple parties while ensuring data privacy. We propose two variants of federated XGBoost with privacy guarantee: FedXGBoost-SMM and FedXGBoost-LDP.
  arXiv  Detail & Related papers  (2021-06-20T09:17:45Z)
• Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data [93.76907759950608]
  We propose a doubly kernel learning algorithm for vertically partitioned data. We show that FDSKL is significantly faster than state-of-the-art federated learning methods when dealing with kernels.
  arXiv  Detail & Related papers  (2020-08-14T05:46:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.