Related papers: The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective

The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective

URL: http://arxiv.org/abs/2405.16918v1
Date: Mon, 27 May 2024 08:10:46 GMT
Title: The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective
Authors: Nils Philipp Walter, Linara Adilova, Jilles Vreeken, Michael Kamp,
Abstract summary: Flatness of the loss surface not only correlates positively with generalization but is also related to adversarial robustness. In this paper, we empirically analyze the relation between adversarial examples and relative flatness with respect to the parameters of one layer.
Score: 34.55229189445268
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Flatness of the loss surface not only correlates positively with generalization but is also related to adversarial robustness, since perturbations of inputs relate non-linearly to perturbations of weights. In this paper, we empirically analyze the relation between adversarial examples and relative flatness with respect to the parameters of one layer. We observe a peculiar property of adversarial examples: during an iterative first-order white-box attack, the flatness of the loss surface measured around the adversarial example first becomes sharper until the label is flipped, but if we keep the attack running it runs into a flat uncanny valley where the label remains flipped. We find this phenomenon across various model architectures and datasets. Our results also extend to large language models (LLMs), but due to the discrete nature of the input space and comparatively weak attacks, the adversarial examples rarely reach a truly flat region. Most importantly, this phenomenon shows that flatness alone cannot explain adversarial robustness unless we can also guarantee the behavior of the function around the examples. We theoretically connect relative flatness to adversarial robustness by bounding the third derivative of the loss surface, underlining the need for flatness in combination with a low global Lipschitz constant for a robust model.

Related papers

Unpacking the Resilience of SNLI Contradiction Examples to Attacks [0.38366697175402226]
We apply the Universal Adversarial Attack to examine the model's vulnerabilities. Our analysis revealed substantial drops in accuracy for the entailment and neutral classes. Fine-tuning the model on an augmented dataset with adversarial examples restored its performance to near-baseline levels.
arXiv Detail & Related papers (2024-12-15T12:47:28Z)
Effect of Ambient-Intrinsic Dimension Gap on Adversarial Vulnerability [13.196433643727792]
We argue that the existence of off-manifold attacks is a natural consequence of the dimension gap between the intrinsic and ambient dimensions of the data. For 2-layer ReLU networks, we prove that even though the dimension gap does not affect generalization performance on samples drawn from the observed data space, it makes the clean-trained model more vulnerable to perturbations in the off-manifold direction.
arXiv Detail & Related papers (2024-03-06T15:41:21Z)
A High Dimensional Statistical Model for Adversarial Training: Geometry and Trade-Offs [23.132536217316073]
We introduce a tractable mathematical model where the interplay between the data and adversarial attacker geometries can be studied. Our main theoretical contribution is an exact description of the sufficient statistics for the adversarial empirical risk minimiser. We show that the the presence of multiple different feature types is crucial to the high complexity performances of adversarial training.
arXiv Detail & Related papers (2024-02-08T13:52:35Z)
Benign Overfitting in Adversarially Robust Linear Classification [91.42259226639837]
"Benign overfitting", where classifiers memorize noisy training data yet still achieve a good generalization performance, has drawn great attention in the machine learning community. We show that benign overfitting indeed occurs in adversarial training, a principled approach to defend against adversarial examples.
arXiv Detail & Related papers (2021-12-31T00:27:31Z)
A Frequency Perspective of Adversarial Robustness [72.48178241090149]
We present a frequency-based understanding of adversarial examples, supported by theoretical and empirical findings. Our analysis shows that adversarial examples are neither in high-frequency nor in low-frequency components, but are simply dataset dependent. We propose a frequency-based explanation for the commonly observed accuracy vs. robustness trade-off.
arXiv Detail & Related papers (2021-10-26T19:12:34Z)
Classification and Adversarial examples in an Overparameterized Linear Model: A Signal Processing Perspective [10.515544361834241]
State-of-the-art deep learning classifiers are highly susceptible to infinitesmal adversarial perturbations. We find that the learned model is susceptible to adversaries in an intermediate regime where classification generalizes but regression does not. Despite the adversarial susceptibility, we find that classification with these features can be easier than the more commonly studied "independent feature" models.
arXiv Detail & Related papers (2021-09-27T17:35:42Z)
The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks [51.1848572349154]
neural network models that perfectly fit noisy data can generalize well to unseen test data. We consider interpolating two-layer linear neural networks trained with gradient flow on the squared loss and derive bounds on the excess risk.
arXiv Detail & Related papers (2021-08-25T22:01:01Z)
Relating Adversarially Robust Generalization to Flat Minima [138.59125287276194]
Adversarial training (AT) has become the de-facto standard to obtain models robust against adversarial examples. We study the relationship between robust generalization and flatness of the robust loss landscape in weight space.
arXiv Detail & Related papers (2021-04-09T15:55:01Z)
Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples [67.23103682776049]
Recent zeroth order hard-label attacks on image classification models have shown comparable performance to their first-order, gradient-level alternatives. It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors. We propose an information-theoretic argument based on a noisy manifold distance oracle, which leaks manifold information through the adversary's gradient estimate.
arXiv Detail & Related papers (2021-03-04T20:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.