Efficient Batch Homomorphic Encryption for Vertically Federated XGBoost

• URL: http://arxiv.org/abs/2112.04261v1
• Date: Wed, 8 Dec 2021 12:41:01 GMT
• Title: Efficient Batch Homomorphic Encryption for Vertically Federated XGBoost
• Authors: Wuxing Xu, Hao Fan, Kaixin Li, Kai Yang
• Abstract summary: In this paper, we study the efficiency problem of adapting widely used XGBoost model in real-world applications to vertical federated learning setting. We propose a novel batch homomorphic encryption method to cut the cost of encryption-related and transmission in nearly half.
• Score: 9.442606239058806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: More and more orgainizations and institutions make efforts on using external data to improve the performance of AI services. To address the data privacy and security concerns, federated learning has attracted increasing attention from both academia and industry to securely construct AI models across multiple isolated data providers. In this paper, we studied the efficiency problem of adapting widely used XGBoost model in real-world applications to vertical federated learning setting. State-of-the-art vertical federated XGBoost frameworks requires large number of encryption operations and ciphertext transmissions, which makes the model training much less efficient than training XGBoost models locally. To bridge this gap, we proposed a novel batch homomorphic encryption method to cut the cost of encryption-related computation and transmission in nearly half. This is achieved by encoding the first-order derivative and the second-order derivative into a single number for encryption, ciphertext transmission, and homomorphic addition operations. The sum of multiple first-order derivatives and second-order derivatives can be simultaneously decoded from the sum of encoded values. We are motivated by the batch idea in the work of BatchCrypt for horizontal federated learning, and design a novel batch method to address the limitations of allowing quite few number of negative numbers. The encode procedure of the proposed batch method consists of four steps, including shifting, truncating, quantizing and batching, while the decoding procedure consists of de-quantization and shifting back. The advantages of our method are demonstrated through theoretical analysis and extensive numerical experiments.

Related papers

• Multi-Token Joint Speculative Decoding for Accelerating Large Language Model Inference [41.93955876156331]
  Large language models (LLMs) have demonstrated their power in various tasks, but their inference incurs significant time and energy costs. speculative decoding uses a smaller model to propose one sequence of tokens, which are subsequently validated in batch by the target large model. Compared with autoregressive decoding, speculative decoding generates the same number of tokens with fewer runs of the large model. An algorithm that has better output perplexity and even better efficiency than speculative decoding can be more useful in practice.
  arXiv  Detail & Related papers  (2024-07-12T23:29:54Z)
• Factor Graph Optimization of Error-Correcting Codes for Belief Propagation Decoding [62.25533750469467]
  Low-Density Parity-Check codes possess several advantages over other families of codes. The proposed approach is shown to outperform the decoding performance of existing popular codes by orders of magnitude.
  arXiv  Detail & Related papers  (2024-06-09T12:08:56Z)
• Learning Linear Block Error Correction Codes [62.25533750469467]
  We propose for the first time a unified encoder-decoder training of binary linear block codes. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient.
  arXiv  Detail & Related papers  (2024-05-07T06:47:12Z)
• Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration [54.897493351694195]
  We propose a novel parallel decoding approach, namely textithidden transfer, which decodes multiple successive tokens simultaneously in a single forward pass. In terms of acceleration metrics, we outperform all the single-model acceleration techniques, including Medusa and Self-Speculative decoding.
  arXiv  Detail & Related papers  (2024-04-18T09:17:06Z)
• Efficient Encoder-Decoder Transformer Decoding for Decomposable Tasks [53.550782959908524]
  We introduce a new configuration for encoder-decoder models that improves efficiency on structured output and decomposable tasks. Our method, prompt-in-decoder (PiD), encodes the input once and decodes the output in parallel, boosting both training and inference efficiency.
  arXiv  Detail & Related papers  (2024-03-19T19:27:23Z)
• Contrastive Decoding Improves Reasoning in Large Language Models [55.16503283583076]
  We show that Contrastive Decoding achieves large out-of-the-box improvements over greedy decoding on a variety of reasoning tasks. We show that Contrastive Decoding leads LLaMA-65B to outperform LLaMA 2, GPT-3.5 and PaLM 2-L on the HellaSwag commonsense reasoning benchmark.
  arXiv  Detail & Related papers  (2023-09-17T00:29:32Z)
• Communication-Efficient Decentralized Federated Learning via One-Bit Compressive Sensing [52.402550431781805]
  Decentralized federated learning (DFL) has gained popularity due to its practicality across various applications. Compared to the centralized version, training a shared model among a large number of nodes in DFL is more challenging. We develop a novel algorithm based on the framework of the inexact alternating direction method (iADM)
  arXiv  Detail & Related papers  (2023-08-31T12:22:40Z)
• Encrypted Dynamic Control exploiting Limited Number of Multiplications and a Method using Ring-LWE based Cryptosystem [0.3749861135832073]
  We present a method to encrypt dynamic controllers that can be implemented through most homomorphic encryption schemes. We propose a customization of the method for Ring-Learning With Errors (Ring-LWE) based cryptosystems.
  arXiv  Detail & Related papers  (2023-07-07T08:24:48Z)
• Graph-Collaborated Auto-Encoder Hashing for Multi-view Binary Clustering [11.082316688429641]
  We propose a hashing algorithm based on auto-encoders for multi-view binary clustering. Specifically, we propose a multi-view affinity graphs learning model with low-rank constraint, which can mine the underlying geometric information from multi-view data. We also design an encoder-decoder paradigm to collaborate the multiple affinity graphs, which can learn a unified binary code effectively.
  arXiv  Detail & Related papers  (2023-01-06T12:43:13Z)
• FFConv: Fast Factorized Neural Network Inference on Encrypted Data [9.868787266501036]
  We propose a low-rank factorization method called FFConv to unify convolution and ciphertext packing. Compared to prior art LoLa and Falcon, our method reduces the inference latency by up to 87% and 12%, respectively.
  arXiv  Detail & Related papers  (2021-02-06T03:10:13Z)
• TEDL: A Text Encryption Method Based on Deep Learning [10.428079716944463]
  This paper proposes a novel text encryption method based on deep learning called TEDL. Results of experiments and relevant analyses show that TEDL performs well for security, efficiency, generality, and has a lower demand for the frequency of key redistribution.
  arXiv  Detail & Related papers  (2020-03-09T11:04:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.