Local Repair of Neural Networks Using Optimization

• URL: http://arxiv.org/abs/2109.14041v1
• Date: Tue, 28 Sep 2021 20:52:26 GMT
• Title: Local Repair of Neural Networks Using Optimization
• Authors: Keyvan Majd, Siyu Zhou, Heni Ben Amor, Georgios Fainekos, and Sriram Sankaranarayanan
• Abstract summary: We propose a framework to repair a pre-trained feed-forward neural network (NN) We formulate the properties as a set of predicates that impose constraints on the output of NN over the target input domain. We demonstrate the application of our framework in bounding an affine transformation, correcting an erroneous NN in classification, and bounding the inputs of a NN controller.
• Score: 13.337627875398393
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a framework to repair a pre-trained feed-forward neural network (NN) to satisfy a set of properties. We formulate the properties as a set of predicates that impose constraints on the output of NN over the target input domain. We define the NN repair problem as a Mixed Integer Quadratic Program (MIQP) to adjust the weights of a single layer subject to the given predicates while minimizing the original loss function over the original training domain. We demonstrate the application of our framework in bounding an affine transformation, correcting an erroneous NN in classification, and bounding the inputs of a NN controller.

Related papers

• Initialization-enhanced Physics-Informed Neural Network with Domain Decomposition (IDPINN) [14.65008276932511]
  We propose a new physics-informed neural network framework, IDPINN, to improve prediction accuracy. We numerically evaluated it on several forward problems and demonstrated the benefits of IDPINN in terms of accuracy.
  arXiv  Detail & Related papers  (2024-06-05T12:03:45Z)
• Neural Network Verification with Branch-and-Bound for General Nonlinearities [63.39918329535165]
  Branch-and-bound (BaB) is among the most effective techniques for neural network (NN) verification. We develop a general framework, named GenBaB, to conduct BaB on general nonlinearities to verify NNs with general architectures. We demonstrate the effectiveness of our GenBaB on verifying a wide range of NNs, including NNs with activation functions such as Sigmoid, Tanh, Sine and GeLU.
  arXiv  Detail & Related papers  (2024-05-31T17:51:07Z)
• N-Adaptive Ritz Method: A Neural Network Enriched Partition of Unity for Boundary Value Problems [1.2200609701777907]
  This work introduces a novel neural network-enriched Partition of Unity (NN-PU) approach for solving boundary value problems via artificial neural networks. The NN enrichment is constructed by combining pre-trained feature-encoded NN blocks with an untrained NN block. The proposed method offers accurate solutions while notably reducing the computational cost compared to the conventional adaptive refinement in the mesh-based methods.
  arXiv  Detail & Related papers  (2024-01-16T18:11:14Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Safety Verification for Neural Networks Based on Set-boundary Analysis [5.487915758677295]
  Neural networks (NNs) are increasingly applied in safety-critical systems such as autonomous vehicles. We propose a set-boundary reachability method to investigate the safety verification problem of NNs from a topological perspective.
  arXiv  Detail & Related papers  (2022-10-09T05:55:37Z)
• Automated Repair of Neural Networks [0.26651200086513094]
  We introduce a framework for repairing unsafe NNs w.r.t. safety specification. Our method is able to search for a new, safe NN representation, by modifying only a few of its weight values. We perform extensive experiments which demonstrate the capability of our proposed framework to yield safe NNs w.r.t.
  arXiv  Detail & Related papers  (2022-07-17T12:42:24Z)
• Adaptive Self-supervision Algorithms for Physics-informed Neural Networks [59.822151945132525]
  Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function. We study the impact of the location of the collocation points on the trainability of these models. We propose a novel adaptive collocation scheme which progressively allocates more collocation points to areas where the model is making higher errors.
  arXiv  Detail & Related papers  (2022-07-08T18:17:06Z)
• On Feature Learning in Neural Networks with Global Convergence Guarantees [49.870593940818715]
  We study the optimization of wide neural networks (NNs) via gradient flow (GF) We show that when the input dimension is no less than the size of the training set, the training loss converges to zero at a linear rate under GF. We also show empirically that, unlike in the Neural Tangent Kernel (NTK) regime, our multi-layer model exhibits feature learning and can achieve better generalization performance than its NTK counterpart.
  arXiv  Detail & Related papers  (2022-04-22T15:56:43Z)
• Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient [62.660451283548724]
  ResiNet is a reinforcement learning framework to discover resilient network topologies against various disasters and attacks. We show that ResiNet achieves a near-optimal resilience gain on multiple graphs while balancing the utility, with a large margin compared to existing approaches.
  arXiv  Detail & Related papers  (2021-10-18T06:14:28Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.