Robust Optimization Framework for Training Shallow Neural Networks Using Reachability Method

• URL: http://arxiv.org/abs/2107.12801v1
• Date: Tue, 27 Jul 2021 13:16:20 GMT
• Title: Robust Optimization Framework for Training Shallow Neural Networks Using Reachability Method
• Authors: Yejiang Yang, Weiming Xiang
• Abstract summary: A robust optimization framework is developed to train shallow neural networks based on reachability analysis of neural networks. It has been shown that the developed robust learning method can provide better robustness against perturbations at the price of loss of training accuracy.
• Score: 1.9798034349981157
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, a robust optimization framework is developed to train shallow neural networks based on reachability analysis of neural networks. To characterize noises of input data, the input training data is disturbed in the description of interval sets. Interval-based reachability analysis is then performed for the hidden layer. With the reachability analysis results, a robust optimization training method is developed in the framework of robust least-square problems. Then, the developed robust least-square problem is relaxed to a semidefinite programming problem. It has been shown that the developed robust learning method can provide better robustness against perturbations at the price of loss of training accuracy to some extent. At last, the proposed method is evaluated on a robot arm model learning example.

Related papers

• A constrained optimization approach to improve robustness of neural networks [1.2338729811609355]
  We present a novel nonlinear programming-based approach to fine-tune pre-trained neural networks to improve robustness against adversarial attacks while maintaining accuracy on clean data.
  arXiv  Detail & Related papers  (2024-09-18T18:37:14Z)
• Maintaining Adversarial Robustness in Continuous Learning [10.746120682014437]
  Adversarial robustness enhanced by defense algorithms is easily erased as the neural network's weights update to learn new tasks. We propose a novel gradient projection technique that effectively stabilizes sample gradients from previous data. This technique can maintaining robustness by collaborating with a class of defense algorithms through sample gradient smoothing.
  arXiv  Detail & Related papers  (2024-02-17T05:14:47Z)
• An Analytic End-to-End Deep Learning Algorithm based on Collaborative Learning [5.710971447109949]
  This paper presents a convergence analysis for end-to-end deep learning of fully connected neural networks (FNN) with smooth activation functions. The proposed method avoids any potential chattering problem, and it also does not easily lead to gradient vanishing problems.
  arXiv  Detail & Related papers  (2023-05-26T08:09:03Z)
• Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systems [5.359060261460183]
  We present a contraction-guided adaptive partitioning algorithm for improving interval-valued reachable set estimates in a nonlinear feedback loop. By leveraging a decoupling of the neural network verification step and reachability partitioning layers, the algorithm can provide accuracy improvements for little computational cost. We report a sizable improvement in the accuracy of reachable set estimation in a fraction of the runtime as compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-07T14:43:21Z)
• A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking [54.89987482509155]
  robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts. We establish a comprehensive benchmark robustness called textbfARES-Bench on the image classification task. By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness.
  arXiv  Detail & Related papers  (2023-02-28T04:26:20Z)
• A Novel Noise Injection-based Training Scheme for Better Model Robustness [9.749718440407811]
  Noise injection-based method has been shown to be able to improve the robustness of artificial neural networks. In this work, we propose a novel noise injection-based training scheme for better model robustness. Experiment results show that our proposed method achieves a much better performance on adversarial robustness and slightly better performance on original accuracy.
  arXiv  Detail & Related papers  (2023-02-17T02:50:25Z)
• Towards Robust Dataset Learning [90.2590325441068]
  We propose a principled, tri-level optimization to formulate the robust dataset learning problem. Under an abstraction model that characterizes robust vs. non-robust features, the proposed method provably learns a robust dataset.
  arXiv  Detail & Related papers  (2022-11-19T17:06:10Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• Improving robustness of jet tagging algorithms with adversarial training [56.79800815519762]
  We investigate the vulnerability of flavor tagging algorithms via application of adversarial attacks. We present an adversarial training strategy that mitigates the impact of such simulated attacks.
  arXiv  Detail & Related papers  (2022-03-25T19:57:19Z)
• Learning Neural Network Subspaces [74.44457651546728]
  Recent observations have advanced our understanding of the neural network optimization landscape. With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks. With a similar computational cost as training one model, we learn lines, curves, and simplexes of high-accuracy neural networks.
  arXiv  Detail & Related papers  (2021-02-20T23:26:58Z)
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning [90.44219200633286]
  We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy. Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-12-25T20:50:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.