Reinforcement Based Learning on Classification Task Could Yield Better
Generalization and Adversarial Accuracy
- URL: http://arxiv.org/abs/2012.04353v1
- Date: Tue, 8 Dec 2020 11:03:17 GMT
- Title: Reinforcement Based Learning on Classification Task Could Yield Better
Generalization and Adversarial Accuracy
- Authors: Shashi Kant Gupta
- Abstract summary: We propose a novel method to train deep learning models on an image classification task.
We use a reward-based optimization function, similar to the vanilla policy gradient method used in reinforcement learning.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Deep Learning has become interestingly popular in computer vision, mostly
attaining near or above human-level performance in various vision tasks. But
recent work has also demonstrated that these deep neural networks are very
vulnerable to adversarial examples (adversarial examples - inputs to a model
which are naturally similar to original data but fools the model in classifying
it into a wrong class). Humans are very robust against such perturbations; one
possible reason could be that humans do not learn to classify based on an error
between "target label" and "predicted label" but possibly due to reinforcements
that they receive on their predictions. In this work, we proposed a novel
method to train deep learning models on an image classification task. We used a
reward-based optimization function, similar to the vanilla policy gradient
method used in reinforcement learning, to train our model instead of
conventional cross-entropy loss. An empirical evaluation on the cifar10 dataset
showed that our method learns a more robust classifier than the same model
architecture trained using cross-entropy loss function (on adversarial
training). At the same time, our method shows a better generalization with the
difference in test accuracy and train accuracy $< 2\%$ for most of the time
compared to the cross-entropy one, whose difference most of the time remains $>
2\%$.
Related papers
- MOREL: Enhancing Adversarial Robustness through Multi-Objective Representation Learning [1.534667887016089]
deep neural networks (DNNs) are vulnerable to slight adversarial perturbations.
We show that strong feature representation learning during training can significantly enhance the original model's robustness.
We propose MOREL, a multi-objective feature representation learning approach, encouraging classification models to produce similar features for inputs within the same class, despite perturbations.
arXiv Detail & Related papers (2024-10-02T16:05:03Z) - Relearning Forgotten Knowledge: on Forgetting, Overfit and Training-Free
Ensembles of DNNs [9.010643838773477]
We introduce a novel score for quantifying overfit, which monitors the forgetting rate of deep models on validation data.
We show that overfit can occur with and without a decrease in validation accuracy, and may be more common than previously appreciated.
We use our observations to construct a new ensemble method, based solely on the training history of a single network, which provides significant improvement without any additional cost in training time.
arXiv Detail & Related papers (2023-10-17T09:22:22Z) - Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases [62.54519787811138]
We present a simple but effective method to measure and mitigate model biases caused by reliance on spurious cues.
We rank images within their classes based on spuriosity, proxied via deep neural features of an interpretable network.
Our results suggest that model bias due to spurious feature reliance is influenced far more by what the model is trained on than how it is trained.
arXiv Detail & Related papers (2022-12-05T23:15:43Z) - X-model: Improving Data Efficiency in Deep Learning with A Minimax Model [78.55482897452417]
We aim at improving data efficiency for both classification and regression setups in deep learning.
To take the power of both worlds, we propose a novel X-model.
X-model plays a minimax game between the feature extractor and task-specific heads.
arXiv Detail & Related papers (2021-10-09T13:56:48Z) - Novelty Detection via Contrastive Learning with Negative Data
Augmentation [34.39521195691397]
We introduce a novel generative network framework for novelty detection.
Our model has significant superiority over cutting-edge novelty detectors.
Our model is more stable for training in a non-adversarial manner, compared to other adversarial based novelty detection methods.
arXiv Detail & Related papers (2021-06-18T07:26:15Z) - An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human
Pose Estimation [80.02124918255059]
Semi-supervised learning aims to boost the accuracy of a model by exploring unlabeled images.
We learn two networks to mutually teach each other.
The more reliable predictions on easy images in each network are used to teach the other network to learn about the corresponding hard images.
arXiv Detail & Related papers (2020-11-25T03:29:52Z) - Learning from Failure: Training Debiased Classifier from Biased
Classifier [76.52804102765931]
We show that neural networks learn to rely on spurious correlation only when it is "easier" to learn than the desired knowledge.
We propose a failure-based debiasing scheme by training a pair of neural networks simultaneously.
Our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets.
arXiv Detail & Related papers (2020-07-06T07:20:29Z) - Adversarial Self-Supervised Contrastive Learning [62.17538130778111]
Existing adversarial learning approaches mostly use class labels to generate adversarial samples that lead to incorrect predictions.
We propose a novel adversarial attack for unlabeled data, which makes the model confuse the instance-level identities of the perturbed data samples.
We present a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data.
arXiv Detail & Related papers (2020-06-13T08:24:33Z) - Learning What Makes a Difference from Counterfactual Examples and
Gradient Supervision [57.14468881854616]
We propose an auxiliary training objective that improves the generalization capabilities of neural networks.
We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task.
Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
arXiv Detail & Related papers (2020-04-20T02:47:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.