Probabilistic Robustness Analysis for DNNs based on PAC Learning

• URL: http://arxiv.org/abs/2101.10102v1
• Date: Mon, 25 Jan 2021 14:10:52 GMT
• Title: Probabilistic Robustness Analysis for DNNs based on PAC Learning
• Authors: Renjue Li and Pengfei Yang and Cheng-Chao Huang and Bai Xue and Lijun Zhang
• Abstract summary: We view a DNN as a function $boldsymbolf$ from inputs to outputs, and consider the local robustness property for a given input. We learn the score difference function $f_i-f_ell$ with respect to the target label $ell$ and attacking label $i$. Our framework can handle very large neural networks like ResNet152 with $6.5$M neurons, and often generates adversarial examples.
• Score: 14.558877524991752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a black box based approach for analysing deep neural networks (DNNs). We view a DNN as a function $\boldsymbol{f}$ from inputs to outputs, and consider the local robustness property for a given input. Based on scenario optimization technique in robust control design, we learn the score difference function $f_i-f_\ell$ with respect to the target label $\ell$ and attacking label $i$. We use a linear template over the input pixels, and learn the corresponding coefficients of the score difference function, based on a reduction to a linear programming (LP) problems. To make it scalable, we propose optimizations including components based learning and focused learning. The learned function offers a probably approximately correct (PAC) guarantee for the robustness property. Since the score difference function is an approximation of the local behaviour of the DNN, it can be used to generate potential adversarial examples, and the original network can be used to check whether they are spurious or not. Finally, we focus on the input pixels with large absolute coefficients, and use them to explain the attacking scenario. We have implemented our approach in a prototypical tool DeepPAC. Our experimental results show that our framework can handle very large neural networks like ResNet152 with $6.5$M neurons, and often generates adversarial examples which are very close to the decision boundary.

Related papers

• Bayesian Inference with Deep Weakly Nonlinear Networks [57.95116787699412]
  We show at a physics level of rigor that Bayesian inference with a fully connected neural network is solvable. We provide techniques to compute the model evidence and posterior to arbitrary order in $1/N$ and at arbitrary temperature.
  arXiv  Detail & Related papers  (2024-05-26T17:08:04Z)
• Bayesian Neural Networks: A Min-Max Game Framework [1.8032347672439046]
  We formulate the BNN via game theory between the deterministic neural network $f$ and the sampling network $f + xi$ or $f + r*xi$. Compared with previous BNN, BNN via game theory learns a solution space within a certain gap between the center $f$ and the sampling point $f + r*xi$. The minimum points between $f$ and $f + r*xi$ become stable when the subspace dimension is large enough with a well-trained model $f$.
  arXiv  Detail & Related papers  (2023-11-18T17:17:15Z)
• The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes [75.59720049837459]
  We study the transition from infinite-width behavior to this variance limited regime as a function of sample size $P$ and network width $N$. We find that finite-size effects can become relevant for very small datasets on the order of $P* sim sqrtN$ for regression with ReLU networks.
  arXiv  Detail & Related papers  (2022-12-23T04:48:04Z)
• Supervised Contrastive Prototype Learning: Augmentation Free Robust Neural Network [17.10753224600936]
  Transformations in the input space of Deep Neural Networks (DNN) lead to unintended changes in the feature space. We propose a training framework, $textbfd Contrastive Prototype Learning$ ( SCPL) We use N-pair contrastive loss with prototypes of the same and opposite classes and replace a categorical classification head with a $textbfPrototype Classification Head$ (PCH) Our approach is $textitsample efficient$, does not require $textitsample mining$, can be implemented on any existing DNN without modification to their
  arXiv  Detail & Related papers  (2022-11-26T01:17:15Z)
• Neural Greedy Pursuit for Feature Selection [72.4121881681861]
  We propose a greedy algorithm to select $N$ important features among $P$ input features for a non-linear prediction problem. We use neural networks as predictors in the algorithm to compute the loss.
  arXiv  Detail & Related papers  (2022-07-19T16:39:16Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• Approximating smooth functions by deep neural networks with sigmoid activation function [0.0]
  We study the power of deep neural networks (DNNs) with sigmoid activation function. We show that DNNs with fixed depth and a width of order $Md$ achieve an approximation rate of $M-2p$.
  arXiv  Detail & Related papers  (2020-10-08T07:29:31Z)
• Deep Polynomial Neural Networks [77.70761658507507]
  $Pi$Nets are a new class of function approximators based on expansions. $Pi$Nets produce state-the-art results in three challenging tasks, i.e. image generation, face verification and 3D mesh representation learning.
  arXiv  Detail & Related papers  (2020-06-20T16:23:32Z)
• Towards Deep Learning Models Resistant to Large Perturbations [0.0]
  Adversarial robustness has proven to be a required property of machine learning algorithms. We show that the well-established algorithm called "adversarial training" fails to train a deep neural network given a large, but reasonable, perturbation magnitude.
  arXiv  Detail & Related papers  (2020-03-30T12:03:09Z)
• Determination of the Semion Code Threshold using Neural Decoders [0.0]
  We compute the error threshold for the semion code, the companion of the Kitaev toric code with the same gauge symmetry group $mathbbZ$. We take advantage of the near-optimal performance of some neural network decoders: multilayer perceptrons and convolutional neural networks.
  arXiv  Detail & Related papers  (2020-02-20T10:56:47Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.