Efficient Encrypted Inference on Ensembles of Decision Trees
- URL: http://arxiv.org/abs/2103.03411v1
- Date: Fri, 5 Mar 2021 01:06:30 GMT
- Title: Efficient Encrypted Inference on Ensembles of Decision Trees
- Authors: Kanthi Sarpatwar and Karthik Nandakumar and Nalini Ratha and James
Rayfield and Karthikeyan Shanmugam and Sharath Pankanti and Roman Vaculin
- Abstract summary: Data privacy concerns often prevent the use of cloud-based machine learning services for sensitive personal data.
We propose a framework to transfer knowledge extracted by complex decision tree ensembles to shallow neural networks.
Our system is highly scalable and can perform efficient inference on batched encrypted (134 bits of security) data with amortized time in milliseconds.
- Score: 21.570003967858355
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data privacy concerns often prevent the use of cloud-based machine learning
services for sensitive personal data. While homomorphic encryption (HE) offers
a potential solution by enabling computations on encrypted data, the challenge
is to obtain accurate machine learning models that work within the
multiplicative depth constraints of a leveled HE scheme. Existing approaches
for encrypted inference either make ad-hoc simplifications to a pre-trained
model (e.g., replace hard comparisons in a decision tree with soft comparators)
at the cost of accuracy or directly train a new depth-constrained model using
the original training set. In this work, we propose a framework to transfer
knowledge extracted by complex decision tree ensembles to shallow neural
networks (referred to as DTNets) that are highly conducive to encrypted
inference. Our approach minimizes the accuracy loss by searching for the best
DTNet architecture that operates within the given depth constraints and
training this DTNet using only synthetic data sampled from the training data
distribution. Extensive experiments on real-world datasets demonstrate that
these characteristics are critical in ensuring that DTNet accuracy approaches
that of the original tree ensemble. Our system is highly scalable and can
perform efficient inference on batched encrypted (134 bits of security) data
with amortized time in milliseconds. This is approximately three orders of
magnitude faster than the standard approach of applying soft comparison at the
internal nodes of the ensemble trees.
Related papers
- A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks [81.2624272756733]
In dense retrieval, deep encoders provide embeddings for both inputs and targets.
We train a small parametric corrector network that adjusts stale cached target embeddings.
Our approach matches state-of-the-art results even when no target embedding updates are made during training.
arXiv Detail & Related papers (2024-09-03T13:29:13Z) - Verifiable Learning for Robust Tree Ensembles [8.207928136395184]
A class of decision tree ensembles called large-spread ensembles admit a security verification algorithm running in restricted time.
We show the benefits of this idea by designing a new training algorithm that automatically learns a large-spread decision tree ensemble from labelled data.
Experimental results on public datasets confirm that large-spread ensembles trained using our algorithm can be verified in a matter of seconds.
arXiv Detail & Related papers (2023-05-05T15:37:23Z) - NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction.
The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network.
A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
arXiv Detail & Related papers (2022-09-29T04:06:00Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - Fast Deep Autoencoder for Federated learning [0.0]
DAEF (Deep Autoencoder for Federated learning) is a novel, fast and privacy preserving implementation of deep autoencoders.
Unlike traditional neural networks, DAEF trains a deep autoencoder network in a non-iterative way, which drastically reduces its training time.
The method has been evaluated and compared to traditional (iterative) deep autoencoders using seven real anomaly detection datasets.
arXiv Detail & Related papers (2022-06-10T14:17:06Z) - Robust Training under Label Noise by Over-parameterization [41.03008228953627]
We propose a principled approach for robust training of over-parameterized deep networks in classification tasks where a proportion of training labels are corrupted.
The main idea is yet very simple: label noise is sparse and incoherent with the network learned from clean data, so we model the noise and learn to separate it from the data.
Remarkably, when trained using such a simple method in practice, we demonstrate state-of-the-art test accuracy against label noise on a variety of real datasets.
arXiv Detail & Related papers (2022-02-28T18:50:10Z) - On Deep Learning with Label Differential Privacy [54.45348348861426]
We study the multi-class classification setting where the labels are considered sensitive and ought to be protected.
We propose a new algorithm for training deep neural networks with label differential privacy, and run evaluations on several datasets.
arXiv Detail & Related papers (2021-02-11T15:09:06Z) - Cryptotree: fast and accurate predictions on encrypted structured data [0.0]
Homomorphic Encryption (HE) is acknowledged for its ability to allow computation on encrypted data, where both the input and output are encrypted.
We propose Cryptotree, a framework that enables the use of Random Forests (RF), a very powerful learning procedure compared to linear regression.
arXiv Detail & Related papers (2020-06-15T11:48:01Z) - Uncertainty Estimation Using a Single Deep Deterministic Neural Network [66.26231423824089]
We propose a method for training a deterministic deep model that can find and reject out of distribution data points at test time with a single forward pass.
We scale training in these with a novel loss function and centroid updating scheme and match the accuracy of softmax models.
arXiv Detail & Related papers (2020-03-04T12:27:36Z) - User-Level Privacy-Preserving Federated Learning: Analysis and
Performance Optimization [77.43075255745389]
Federated learning (FL) is capable of preserving private data from mobile terminals (MTs) while training the data into useful models.
From a viewpoint of information theory, it is still possible for a curious server to infer private information from the shared models uploaded by MTs.
We propose a user-level differential privacy (UDP) algorithm by adding artificial noise to the shared models before uploading them to servers.
arXiv Detail & Related papers (2020-02-29T10:13:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.