Towards Accurate and Robust Architectures via Neural Architecture Search

• URL: http://arxiv.org/abs/2405.05502v1
• Date: Thu, 9 May 2024 02:16:50 GMT
• Title: Towards Accurate and Robust Architectures via Neural Architecture Search
• Authors: Yuwei Ou, Yuqi Feng, Yanan Sun,
• Abstract summary: adversarial training improves accuracy and robustness by adjusting the weight connection affiliated to the architecture. We propose ARNAS to search for accurate and robust architectures for adversarial training.
• Score: 3.4014222238829497
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To defend deep neural networks from adversarial attacks, adversarial training has been drawing increasing attention for its effectiveness. However, the accuracy and robustness resulting from the adversarial training are limited by the architecture, because adversarial training improves accuracy and robustness by adjusting the weight connection affiliated to the architecture. In this work, we propose ARNAS to search for accurate and robust architectures for adversarial training. First we design an accurate and robust search space, in which the placement of the cells and the proportional relationship of the filter numbers are carefully determined. With the design, the architectures can obtain both accuracy and robustness by deploying accurate and robust structures to their sensitive positions, respectively. Then we propose a differentiable multi-objective search strategy, performing gradient descent towards directions that are beneficial for both natural loss and adversarial loss, thus the accuracy and robustness can be guaranteed at the same time. We conduct comprehensive experiments in terms of white-box attacks, black-box attacks, and transferability. Experimental results show that the searched architecture has the strongest robustness with the competitive accuracy, and breaks the traditional idea that NAS-based architectures cannot transfer well to complex tasks in robustness scenarios. By analyzing outstanding architectures searched, we also conclude that accurate and robust neural architectures tend to deploy different structures near the input and output, which has great practical significance on both hand-crafting and automatically designing of accurate and robust architectures.

Related papers

• Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness [32.914986455418]
  We propose a Reinforced Compressive Neural Architecture Search (RC-NAS) for Versatile Adversarial Robustness. Specifically, we define task settings that compose datasets, adversarial attacks, and teacher network information. Experiments show that our framework could achieve adaptive compression towards different initial teacher networks, datasets, and adversarial attacks.
  arXiv  Detail & Related papers  (2024-06-10T20:59:52Z)
• Robust NAS under adversarial training: benchmark, theory, and beyond [55.51199265630444]
  We release a comprehensive data set that encompasses both clean accuracy and robust accuracy for a vast array of adversarially trained networks. We also establish a generalization theory for searching architecture in terms of clean accuracy and robust accuracy under multi-objective adversarial training.
  arXiv  Detail & Related papers  (2024-03-19T20:10:23Z)
• Neural Architecture Design and Robustness: A Dataset [11.83842808044211]
  We introduce a database on neural architecture design and robustness evaluations. We evaluate all these networks on a range of common adversarial attacks and corruption types. We find that carefully crafting the topology of a network can have substantial impact on its robustness.
  arXiv  Detail & Related papers  (2023-06-11T16:02:14Z)
• A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking [54.89987482509155]
  robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts. We establish a comprehensive benchmark robustness called textbfARES-Bench on the image classification task. By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness.
  arXiv  Detail & Related papers  (2023-02-28T04:26:20Z)
• Differentiable Search of Accurate and Robust Architectures [22.435774101990752]
  adversarial training has been drawing increasing attention because of its simplicity and effectiveness. Deep neural networks (DNNs) are found to be vulnerable to adversarial attacks. We propose DSARA to automatically search for the neural architectures that are accurate and robust after adversarial training.
  arXiv  Detail & Related papers  (2022-12-28T08:36:36Z)
• Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks [98.21130211336964]
  Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. In this paper, we investigate the impact of network width and depth on the robustness of adversarially trained DNNs.
  arXiv  Detail & Related papers  (2021-10-07T23:13:33Z)
• RobustART: Benchmarking Robustness on Architecture Design and Training Techniques [170.3297213957074]
  Deep neural networks (DNNs) are vulnerable to adversarial noises. There are no comprehensive studies of how architecture design and training techniques affect robustness. We propose the first comprehensiveness investigation benchmark on ImageNet.
  arXiv  Detail & Related papers  (2021-09-11T08:01:14Z)
• Neural Architecture Dilation for Adversarial Robustness [56.18555072877193]
  A shortcoming of convolutional neural networks is that they are vulnerable to adversarial attacks. This paper aims to improve the adversarial robustness of the backbone CNNs that have a satisfactory accuracy. Under a minimal computational overhead, a dilation architecture is expected to be friendly with the standard performance of the backbone CNN.
  arXiv  Detail & Related papers  (2021-08-16T03:58:00Z)
• DSRNA: Differentiable Search of Robust Neural Architectures [11.232234265070753]
  In deep learning applications, the architectures of deep neural networks are crucial in achieving high accuracy. We propose methods to perform differentiable search of robust neural architectures. Our methods are more robust to various norm-bound attacks than several robust NAS baselines.
  arXiv  Detail & Related papers  (2020-12-11T04:52:54Z)
• On Adversarial Robustness: A Neural Architecture Search perspective [20.478741635006113]
  This work is the first large-scale study to understand adversarial robustness purely from an architectural perspective. We show that random sampling in the search space of DARTS with simple ensembling can improve the robustness to PGD attack by nearly12%. We show that NAS, which is popular for achieving SoTA accuracy, can provide adversarial accuracy as a free add-on without any form of adversarial training.
  arXiv  Detail & Related papers  (2020-07-16T16:07:10Z)
• Neural Ensemble Search for Uncertainty Estimation and Dataset Shift [67.57720300323928]
  Ensembles of neural networks achieve superior performance compared to stand-alone networks in terms of accuracy, uncertainty calibration and robustness to dataset shift. We propose two methods for automatically constructing ensembles with emphvarying architectures. We show that the resulting ensembles outperform deep ensembles not only in terms of accuracy but also uncertainty calibration and robustness to dataset shift.
  arXiv  Detail & Related papers  (2020-06-15T17:38:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.