Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation

• URL: http://arxiv.org/abs/2311.17607v2
• Date: Mon, 19 Aug 2024 11:26:40 GMT
• Title: Topology-preserving Adversarial Training for Alleviating Natural Accuracy Degradation
• Authors: Xiaoyue Mi, Fan Tang, Yepeng Weng, Danding Wang, Juan Cao, Sheng Tang, Peng Li, Yang Liu,
• Abstract summary: Adversarial training has suffered from the natural accuracy degradation problem. We propose Topology-pReserving Adversarial traINing (TRAIN) to alleviate the problem. We show TRAIN achieves up to 8.86% improvement in natural accuracy and 6.33% improvement in robust accuracy.
• Score: 27.11004064848789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the effectiveness in improving the robustness of neural networks, adversarial training has suffered from the natural accuracy degradation problem, i.e., accuracy on natural samples has reduced significantly. In this study, we reveal that natural accuracy degradation is highly related to the disruption of the natural sample topology in the representation space by quantitative and qualitative experiments. Based on this observation, we propose Topology-pReserving Adversarial traINing (TRAIN) to alleviate the problem by preserving the topology structure of natural samples from a standard model trained only on natural samples during adversarial training. As an additional regularization, our method can be combined with various popular adversarial training algorithms, taking advantage of both sides. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet show that our proposed method achieves consistent and significant improvements over various strong baselines in most cases. Specifically, without additional data, TRAIN achieves up to 8.86% improvement in natural accuracy and 6.33% improvement in robust accuracy.

Related papers

• Towards Reliable Evaluation of Neural Program Repair with Natural Robustness Testing [2.763736939516234]
  We first examine the naturalness of semantic-preserving transformations through a two-stage human study. Next, we conduct natural robustness testing on NPR techniques to assess their true effectiveness against real-world data variations.
  arXiv  Detail & Related papers  (2024-02-19T07:07:44Z)
• MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers [41.56951365163419]
  "MixedNUTS" is a training-free method where the output logits of a robust classifier are processed by nonlinear transformations with only three parameters. MixedNUTS then converts the transformed logits into probabilities and mixes them as the overall output. On CIFAR-10, CIFAR-100, and ImageNet datasets, experimental results with custom strong adaptive attacks demonstrate MixedNUTS's vastly improved accuracy and near-SOTA robustness.
  arXiv  Detail & Related papers  (2024-02-03T21:12:36Z)
• Splitting the Difference on Adversarial Training [13.470640587945057]
  adversarial training is one of the most effective defenses against adversarial examples. In this work, we take a fundamentally different approach by treating the perturbed examples of each class as a separate class to be learned. This split doubles the number of classes to be learned, but at the same time considerably simplifies the decision boundaries.
  arXiv  Detail & Related papers  (2023-10-03T23:09:47Z)
• Explicit Tradeoffs between Adversarial and Natural Distributional Robustness [48.44639585732391]
  In practice, models need to enjoy both types of robustness to ensure reliability. In this work, we show that in fact, explicit tradeoffs exist between adversarial and natural distributional robustness.
  arXiv  Detail & Related papers  (2022-09-15T19:58:01Z)
• RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustness [94.69774317059122]
  We show that the effectiveness of the well celebrated Mixup can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss. This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup.
  arXiv  Detail & Related papers  (2022-06-29T09:44:33Z)
• Adversarial Feature Stacking for Accurate and Robust Predictions [4.208059346198116]
  Adversarial Feature Stacking (AFS) model can jointly take advantage of features with varied levels of robustness and accuracy. We evaluate the AFS model on CIFAR-10 and CIFAR-100 datasets with strong adaptive attack methods.
  arXiv  Detail & Related papers  (2021-03-24T12:01:24Z)
• Natural Perturbed Training for General Robustness of Neural Network Classifiers [0.0]
  Natural perturbed learning show better and much faster performance than adversarial training on clean, adversarial as well as natural perturbed images. For Cifar-10 and STL-10 natural perturbed training even improves the accuracy for clean data and reaches the state of the art performance.
  arXiv  Detail & Related papers  (2021-03-21T11:47:38Z)
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning [90.44219200633286]
  We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy. Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
  arXiv  Detail & Related papers  (2020-12-25T20:50:15Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• To be Robust or to be Fair: Towards Fairness in Adversarial Training [83.42241071662897]
  We find that adversarial training algorithms tend to introduce severe disparity of accuracy and robustness between different groups of data. We propose a Fair-Robust-Learning (FRL) framework to mitigate this unfairness problem when doing adversarial defenses.
  arXiv  Detail & Related papers  (2020-10-13T02:21:54Z)
• Adversarial Robustness on In- and Out-Distribution Improves Explainability [109.68938066821246]
  RATIO is a training procedure for robustness via Adversarial Training on In- and Out-distribution. RATIO achieves state-of-the-art $l$-adrial on CIFAR10 and maintains better clean accuracy.
  arXiv  Detail & Related papers  (2020-03-20T18:57:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.