Adversarial training with informed data selection
- URL: http://arxiv.org/abs/2301.04472v1
- Date: Sat, 7 Jan 2023 12:09:50 GMT
- Title: Adversarial training with informed data selection
- Authors: Marcele O. K. Mendon\c{c}a, Javier Maroto, Pascal Frossard and Paulo
S. R. Diniz
- Abstract summary: Adrial training is the most efficient solution to defend the network against these malicious attacks.
This work proposes a data selection strategy to be applied in the mini-batch training.
The simulation results show that a good compromise can be obtained regarding robustness and standard accuracy.
- Score: 53.19381941131439
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: With the increasing amount of available data and advances in computing
capabilities, deep neural networks (DNNs) have been successfully employed to
solve challenging tasks in various areas, including healthcare, climate, and
finance. Nevertheless, state-of-the-art DNNs are susceptible to
quasi-imperceptible perturbed versions of the original images -- adversarial
examples. These perturbations of the network input can lead to disastrous
implications in critical areas where wrong decisions can directly affect human
lives. Adversarial training is the most efficient solution to defend the
network against these malicious attacks. However, adversarial trained networks
generally come with lower clean accuracy and higher computational complexity.
This work proposes a data selection (DS) strategy to be applied in the
mini-batch training. Based on the cross-entropy loss, the most relevant samples
in the batch are selected to update the model parameters in the
backpropagation. The simulation results show that a good compromise can be
obtained regarding robustness and standard accuracy, whereas the computational
complexity of the backpropagation pass is reduced.
Related papers
- Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents.
Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
arXiv Detail & Related papers (2024-04-04T06:24:11Z) - Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks [28.97655735976179]
ReLU networks have been shown to almost always yield high confidence predictions when the test data are far away from the training set.
We overcome this problem by adding a term to the output of the neural network that corresponds to the logit of an extra class.
This technique provably prevents arbitrarily high confidence on far-away test data while maintaining a simple discriminative point-estimate training.
arXiv Detail & Related papers (2023-11-07T03:19:16Z) - Comprehensive Analysis of Network Robustness Evaluation Based on Convolutional Neural Networks with Spatial Pyramid Pooling [4.366824280429597]
Connectivity robustness, a crucial aspect for understanding, optimizing, and repairing complex networks, has traditionally been evaluated through simulations.
We address these challenges by designing a convolutional neural networks (CNN) model with spatial pyramid pooling networks (SPP-net)
We show that the proposed CNN model consistently achieves accurate evaluations of both attack curves and robustness values across all removal scenarios.
arXiv Detail & Related papers (2023-08-10T09:54:22Z) - DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning [5.2319020651074215]
We propose a Curricumum-guided Contrastive Learning framework for neural Predictor (DCLP)
Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution.
We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors.
arXiv Detail & Related papers (2023-02-25T08:16:21Z) - SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural
Networks [56.35403810762512]
Spiking neural networks (SNNs) with event-based computation are promising brain-inspired models for energy-efficient applications on neuromorphic hardware.
We study spike-based implicit differentiation on the equilibrium state (SPIDE) that extends the recently proposed training method.
arXiv Detail & Related papers (2023-02-01T04:22:59Z) - Distributed Adversarial Training to Robustify Deep Neural Networks at
Scale [100.19539096465101]
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification.
To defend against such attacks, an effective approach, known as adversarial training (AT), has been shown to mitigate robust training.
We propose a large-batch adversarial training framework implemented over multiple machines.
arXiv Detail & Related papers (2022-06-13T15:39:43Z) - S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural
Networks via Guided Distribution Calibration [74.5509794733707]
We present a novel guided learning paradigm from real-valued to distill binary networks on the final prediction distribution.
Our proposed method can boost the simple contrastive learning baseline by an absolute gain of 5.515% on BNNs.
Our method achieves substantial improvement over the simple contrastive learning baseline, and is even comparable to many mainstream supervised BNN methods.
arXiv Detail & Related papers (2021-02-17T18:59:28Z) - A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via
Adversarial Fine-tuning [90.44219200633286]
We propose a simple yet very effective adversarial fine-tuning approach based on a $textitslow start, fast decay$ learning rate scheduling strategy.
Experimental results show that the proposed adversarial fine-tuning approach outperforms the state-of-the-art methods on CIFAR-10, CIFAR-100 and ImageNet datasets.
arXiv Detail & Related papers (2020-12-25T20:50:15Z) - Dynamic Hard Pruning of Neural Networks at the Edge of the Internet [11.605253906375424]
Dynamic Hard Pruning (DynHP) technique incrementally prunes the network during training.
DynHP enables a tunable size reduction of the final neural network and reduces the NN memory occupancy during training.
Freed memory is reused by a emphdynamic batch sizing approach to counterbalance the accuracy degradation caused by the hard pruning strategy.
arXiv Detail & Related papers (2020-11-17T10:23:28Z) - Initializing Perturbations in Multiple Directions for Fast Adversarial
Training [1.8638865257327277]
In image classification, an adversarial example can fool the well trained deep neural networks by adding barely imperceptible perturbations to clean images.
Adversarial Training, one of the most direct and effective methods, minimizes the losses of perturbed-data.
We propose the Diversified Initialized Perturbations Adversarial Training (DIP-FAT)
arXiv Detail & Related papers (2020-05-15T15:52:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.