A Domain-Theoretic Framework for Robustness Analysis of Neural Networks

• URL: http://arxiv.org/abs/2203.00295v1
• Date: Tue, 1 Mar 2022 09:01:01 GMT
• Title: A Domain-Theoretic Framework for Robustness Analysis of Neural Networks
• Authors: Can Zhou, Razin A. Shaikh, Yiran Li, Amin Farjudian
• Abstract summary: We present a domain-theoretic framework for validated robustness analysis of neural networks. We develop a validated algorithm for estimation of Lipschitz constant of feedforward regressors. Within our domain model, differentiable and non-differentiable networks can be analyzed uniformly.
• Score: 2.425920001184443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a domain-theoretic framework for validated robustness analysis of neural networks. We first analyze the global robustness of a general class of networks. Then, using the fact that, over finite-dimensional Banach spaces, the domain-theoretic L-derivative coincides with Clarke's generalized gradient, we extend our framework for attack-agnostic local robustness analysis. Our framework is ideal for designing algorithms which are correct by construction. We exemplify this claim by developing a validated algorithm for estimation of Lipschitz constant of feedforward regressors. We prove the completeness of the algorithm over differentiable networks, and also over general position ReLU networks. Within our domain model, differentiable and non-differentiable networks can be analyzed uniformly. We implement our algorithm using arbitrary-precision interval arithmetic, and present the results of some experiments. Our implementation is truly validated, as it handles floating-point errors as well.

Related papers

• A Unified Analysis of Generalization and Sample Complexity for Semi-Supervised Domain Adaptation [1.9567015559455132]
  Domain adaptation seeks to leverage the abundant label information in a source domain to improve classification performance in a target domain with limited labels.<n>Most existing theoretical analyses focus on simplified settings where the source and target domains share the same input space.<n>We present a comprehensive theoretical study of domain adaptation algorithms based on domain alignment.
  arXiv  Detail & Related papers  (2025-07-30T12:53:08Z)
• Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality [52.906438147288256]
  We show that our algorithm can identify the globally optimal reward and policy under certain neural network structures. This is the first IRL algorithm with a non-asymptotic convergence guarantee that provably achieves global optimality.
  arXiv  Detail & Related papers  (2025-03-22T21:16:08Z)
• A Globally Convergent Algorithm for Neural Network Parameter Optimization Based on Difference-of-Convex Functions [29.58728073957055]
  We propose an algorithm for optimizing parameters of hidden layer networks. Specifically, we derive a blockwise (DC-of-the-art) difference function.
  arXiv  Detail & Related papers  (2024-01-15T19:53:35Z)
• Does a sparse ReLU network training problem always admit an optimum? [0.0]
  We show that the existence of an optimal solution is not always guaranteed, especially in the context of sparse ReLU neural networks. In particular, we first show that optimization problems involving deep networks with certain sparsity patterns do not always have optimal parameters.
  arXiv  Detail & Related papers  (2023-06-05T08:01:50Z)
• Structured Optimal Variational Inference for Dynamic Latent Space Models [16.531262817315696]
  We consider a latent space model for dynamic networks, where our objective is to estimate the pairwise inner products plus the intercept of the latent positions. To balance posterior inference and computational scalability, we consider a structured mean-field variational inference framework.
  arXiv  Detail & Related papers  (2022-09-29T22:10:42Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Robust Learning of Parsimonious Deep Neural Networks [0.0]
  We propose a simultaneous learning and pruning algorithm capable of identifying and eliminating irrelevant structures in a neural network. We derive a novel hyper-prior distribution over the prior parameters that is crucial for their optimal selection. We evaluate the proposed algorithm on the MNIST data set and commonly used fully connected and convolutional LeNet architectures.
  arXiv  Detail & Related papers  (2022-05-10T03:38:55Z)
• Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks [75.33431791218302]
  We study the training problem of deep neural networks and introduce an analytic approach to unveil hidden convexity in the optimization landscape. We consider a deep parallel ReLU network architecture, which also includes standard deep networks and ResNets as its special cases.
  arXiv  Detail & Related papers  (2021-10-18T18:00:36Z)
• Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent [95.94432031144716]
  We propose a unified non- optimization framework for the analysis of a learning network. We show that existing guarantees can be trained unified through gradient descent.
  arXiv  Detail & Related papers  (2021-06-25T17:45:00Z)
• Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms [71.62575565990502]
  We prove that the generalization error of an optimization algorithm can be bounded on the complexity' of the fractal structure that underlies its generalization measure. We further specialize our results to specific problems (e.g., linear/logistic regression, one hidden/layered neural networks) and algorithms.
  arXiv  Detail & Related papers  (2021-06-09T08:05:36Z)
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty [111.22588110362705]
  Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference. We derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network.
  arXiv  Detail & Related papers  (2021-05-09T21:33:27Z)
• A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks [56.084798078072396]
  We take a step towards closing the gap between theory and practice by significantly improving the known theoretical bounds on both the network width and the convergence time. We show that convergence to a global minimum is guaranteed for networks with quadratic widths in the sample size and linear in their depth at a time logarithmic in both. Our analysis and convergence bounds are derived via the construction of a surrogate network with fixed activation patterns that can be transformed at any time to an equivalent ReLU network of a reasonable size.
  arXiv  Detail & Related papers  (2021-01-12T00:40:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.