Verification of Neural Networks Local Differential Classification
Privacy
- URL: http://arxiv.org/abs/2310.20299v1
- Date: Tue, 31 Oct 2023 09:11:12 GMT
- Title: Verification of Neural Networks Local Differential Classification
Privacy
- Authors: Roie Reshef, Anan Kabaha, Olga Seleznova, and Dana Drachsler-Cohen
- Abstract summary: We propose a new privacy property, called local differential classification privacy (LDCP)
LDCP extends local robustness to a differential privacy setting suitable for black-box classifiers.
We propose Sphynx, an algorithm that computes an abstraction of all networks, with a high probability, from a small set of networks.
- Score: 1.3024517678456733
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Neural networks are susceptible to privacy attacks. To date, no verifier can
reason about the privacy of individuals participating in the training set. We
propose a new privacy property, called local differential classification
privacy (LDCP), extending local robustness to a differential privacy setting
suitable for black-box classifiers. Given a neighborhood of inputs, a
classifier is LDCP if it classifies all inputs the same regardless of whether
it is trained with the full dataset or whether any single entry is omitted. A
naive algorithm is highly impractical because it involves training a very large
number of networks and verifying local robustness of the given neighborhood
separately for every network. We propose Sphynx, an algorithm that computes an
abstraction of all networks, with a high probability, from a small set of
networks, and verifies LDCP directly on the abstract network. The challenge is
twofold: network parameters do not adhere to a known distribution probability,
making it difficult to predict an abstraction, and predicting too large
abstraction harms the verification. Our key idea is to transform the parameters
into a distribution given by KDE, allowing to keep the over-approximation error
small. To verify LDCP, we extend a MILP verifier to analyze an abstract
network. Experimental results show that by training only 7% of the networks,
Sphynx predicts an abstract network obtaining 93% verification accuracy and
reducing the analysis time by $1.7\cdot10^4$x.
Related papers
- VeriFlow: Modeling Distributions for Neural Network Verification [4.3012765978447565]
Formal verification has emerged as a promising method to ensure the safety and reliability of neural networks.
We propose the VeriFlow architecture as a flow based density model tailored to allow any verification approach to restrict its search to the some data distribution of interest.
arXiv Detail & Related papers (2024-06-20T12:41:39Z) - DP-DCAN: Differentially Private Deep Contrastive Autoencoder Network for Single-cell Clustering [29.96339380816541]
Deep learning models may leak sensitive information about users.
Differential Privacy (DP) is increasingly used to protect privacy.
In this paper, we take advantage of the uniqueness of the autoencoder that it outputs only the dimension-reduced vector in the middle of the network.
We design a Differentially Private Deep Contrastive Autoencoder Network (DP-DCAN) by partial network perturbation for single-cell clustering.
arXiv Detail & Related papers (2023-11-06T05:13:29Z) - Unfolding Local Growth Rate Estimates for (Almost) Perfect Adversarial
Detection [22.99930028876662]
Convolutional neural networks (CNN) define the state-of-the-art solution on many perceptual tasks.
Current CNN approaches largely remain vulnerable against adversarial perturbations of the input that have been crafted specifically to fool the system.
We propose a simple and light-weight detector, which leverages recent findings on the relation between networks' local intrinsic dimensionality (LID) and adversarial attacks.
arXiv Detail & Related papers (2022-12-13T17:51:32Z) - Interpreting deep learning output for out-of-distribution detection [0.6091702876917279]
We develop a new method for out-of-distribution detection in deep learning networks.
The method offers an explanatory step towards understanding and interpretation of the model learning process and its output.
We demonstrate our OOD detection method on a challenging transmission electron microscopy virus image dataset.
arXiv Detail & Related papers (2022-11-07T15:48:08Z) - Robust-by-Design Classification via Unitary-Gradient Neural Networks [66.17379946402859]
The use of neural networks in safety-critical systems requires safe and robust models, due to the existence of adversarial attacks.
Knowing the minimal adversarial perturbation of any input x, or, equivalently, the distance of x from the classification boundary, allows evaluating the classification robustness, providing certifiable predictions.
A novel network architecture named Unitary-Gradient Neural Network is presented.
Experimental results show that the proposed architecture approximates a signed distance, hence allowing an online certifiable classification of x at the cost of a single inference.
arXiv Detail & Related papers (2022-09-09T13:34:51Z) - DAAIN: Detection of Anomalous and Adversarial Input using Normalizing
Flows [52.31831255787147]
We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA)
Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution.
Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
arXiv Detail & Related papers (2021-05-30T22:07:13Z) - Full network nonlocality [68.8204255655161]
We introduce the concept of full network nonlocality, which describes correlations that necessitate all links in a network to distribute nonlocal resources.
We show that the most well-known network Bell test does not witness full network nonlocality.
More generally, we point out that established methods for analysing local and theory-independent correlations in networks can be combined in order to deduce sufficient conditions for full network nonlocality.
arXiv Detail & Related papers (2021-05-19T18:00:02Z) - Enabling certification of verification-agnostic networks via
memory-efficient semidefinite programming [97.40955121478716]
We propose a first-order dual SDP algorithm that requires memory only linear in the total number of network activations.
We significantly improve L-inf verified robust accuracy from 1% to 88% and 6% to 40% respectively.
We also demonstrate tight verification of a quadratic stability specification for the decoder of a variational autoencoder.
arXiv Detail & Related papers (2020-10-22T12:32:29Z) - Cassandra: Detecting Trojaned Networks from Adversarial Perturbations [92.43879594465422]
In many cases, pre-trained models are sourced from vendors who may have disrupted the training pipeline to insert Trojan behaviors into the models.
We propose a method to verify if a pre-trained model is Trojaned or benign.
Our method captures fingerprints of neural networks in the form of adversarial perturbations learned from the network gradients.
arXiv Detail & Related papers (2020-07-28T19:00:40Z) - ESPN: Extremely Sparse Pruned Networks [50.436905934791035]
We show that a simple iterative mask discovery method can achieve state-of-the-art compression of very deep networks.
Our algorithm represents a hybrid approach between single shot network pruning methods and Lottery-Ticket type approaches.
arXiv Detail & Related papers (2020-06-28T23:09:27Z) - One-vs-Rest Network-based Deep Probability Model for Open Set
Recognition [6.85316573653194]
An intelligent self-learning system should be able to differentiate between known and unknown examples.
One-vs-rest networks can provide more informative hidden representations for unknown examples than the commonly used SoftMax layer.
The proposed probability model outperformed the state-of-the art methods in open set classification scenarios.
arXiv Detail & Related papers (2020-04-17T05:24:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.