Hyperparameter Optimization for SecureBoost via Constrained Multi-Objective Federated Learning

• URL: http://arxiv.org/abs/2404.04490v1
• Date: Sat, 6 Apr 2024 03:46:42 GMT
• Title: Hyperparameter Optimization for SecureBoost via Constrained Multi-Objective Federated Learning
• Authors: Yan Kang, Ziyao Ren, Lixin Fan, Linghua Yang, Yongxin Tong, Qiang Yang,
• Abstract summary: We find that SecureBoost and some of its variants are still vulnerable to label leakage. This vulnerability may lead to a suboptimal trade-off between utility, privacy, and efficiency. We propose the Constrained Multi-Objective SecureBoost (CMOSB) algorithm to approximate optimal solutions.
• Score: 26.00375717103131
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: SecureBoost is a tree-boosting algorithm that leverages homomorphic encryption (HE) to protect data privacy in vertical federated learning. SecureBoost and its variants have been widely adopted in fields such as finance and healthcare. However, the hyperparameters of SecureBoost are typically configured heuristically for optimizing model performance (i.e., utility) solely, assuming that privacy is secured. Our study found that SecureBoost and some of its variants are still vulnerable to label leakage. This vulnerability may lead the current heuristic hyperparameter configuration of SecureBoost to a suboptimal trade-off between utility, privacy, and efficiency, which are pivotal elements toward a trustworthy federated learning system. To address this issue, we propose the Constrained Multi-Objective SecureBoost (CMOSB) algorithm, which aims to approximate Pareto optimal solutions that each solution is a set of hyperparameters achieving an optimal trade-off between utility loss, training cost, and privacy leakage. We design measurements of the three objectives, including a novel label inference attack named instance clustering attack (ICA) to measure the privacy leakage of SecureBoost. Additionally, we provide two countermeasures against ICA. The experimental results demonstrate that the CMOSB yields superior hyperparameters over those optimized by grid search and Bayesian optimization regarding the trade-off between utility loss, training cost, and privacy leakage.

Related papers

• TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data [50.797729676285876]
  We propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm.
  arXiv  Detail & Related papers  (2024-02-16T16:41:14Z)
• PROFL: A Privacy-Preserving Federated Learning Method with Stringent Defense Against Poisoning Attacks [2.6487166137163007]
  Federated Learning (FL) faces two major issues: privacy leakage and poisoning attacks. We propose a novel privacy-preserving Byzantine-robust FL framework PROFL. PROFL is based on the two-trapdoor additional homomorphic encryption algorithm and blinding techniques.
  arXiv  Detail & Related papers  (2023-12-02T06:34:37Z)
• SecureBoost Hyperparameter Tuning via Multi-Objective Federated Learning [23.196686101682737]
  SecureBoost is a tree-boosting algorithm leveraging homomorphic encryption to protect data privacy in vertical federated learning setting. SecureBoost suffers from high computational complexity and risk of label leakage. We propose a Constrained Multi-Objective SecureBoost (CMOSB) algorithm to find optimal solutions.
  arXiv  Detail & Related papers  (2023-07-20T04:45:59Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• Optimizing Privacy, Utility and Efficiency in Constrained Multi-Objective Federated Learning [20.627157142499378]
  We develop two improved CMOFL algorithms based on NSGA-II and PSL. We design specific measurements of privacy leakage, utility loss, and training cost for three privacy protection mechanisms. Empirical experiments conducted under each of the three protection mechanisms demonstrate the effectiveness of our proposed algorithms.
  arXiv  Detail & Related papers  (2023-04-29T17:55:38Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)
• Decentralized Stochastic Optimization with Inherent Privacy Protection [103.62463469366557]
  Decentralized optimization is the basic building block of modern collaborative machine learning, distributed estimation and control, and large-scale sensing. Since involved data, privacy protection has become an increasingly pressing need in the implementation of decentralized optimization algorithms.
  arXiv  Detail & Related papers  (2022-05-08T14:38:23Z)
• SecureBoost+: Large Scale and High-Performance Vertical Federated Gradient Boosting Decision Tree [18.65547577691255]
  We propose SecureBoost+, a large-scale and high-performance vertical federated gradient boosting decision tree framework. SecureBoost+ is secure in the semi-honest model, which is the same as SecureBoost. The experimental results show that SecureBoost+ is 6-35x faster than SecureBoost, but with the same accuracy and can be scaled up to tens of millions of data samples and thousands of feature dimensions.
  arXiv  Detail & Related papers  (2021-10-21T06:49:10Z)
• Boosting for Online Convex Optimization [64.15578413206715]
  We consider the decision-making framework of online convex optimization with a large number of experts. We define a weak learning algorithm as a mechanism that guarantees approximate regret against a base class of experts. We give an efficient boosting algorithm that guarantees near-optimal regret against the convex hull of the base class.
  arXiv  Detail & Related papers  (2021-02-18T12:30:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.