Hard-label Manifolds: Unexpected Advantages of Query Efficiency for
Finding On-manifold Adversarial Examples
- URL: http://arxiv.org/abs/2103.03325v1
- Date: Thu, 4 Mar 2021 20:53:06 GMT
- Title: Hard-label Manifolds: Unexpected Advantages of Query Efficiency for
Finding On-manifold Adversarial Examples
- Authors: Washington Garcia, Pin-Yu Chen, Somesh Jha, Scott Clouse, Kevin R. B.
Butler
- Abstract summary: Recent zeroth order hard-label attacks on image classification models have shown comparable performance to their first-order, gradient-level alternatives.
It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors.
We propose an information-theoretic argument based on a noisy manifold distance oracle, which leaks manifold information through the adversary's gradient estimate.
- Score: 67.23103682776049
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Designing deep networks robust to adversarial examples remains an open
problem. Likewise, recent zeroth order hard-label attacks on image
classification models have shown comparable performance to their first-order,
gradient-level alternatives. It was recently shown in the gradient-level
setting that regular adversarial examples leave the data manifold, while their
on-manifold counterparts are in fact generalization errors. In this paper, we
argue that query efficiency in the zeroth-order setting is connected to an
adversary's traversal through the data manifold. To explain this behavior, we
propose an information-theoretic argument based on a noisy manifold distance
oracle, which leaks manifold information through the adversary's gradient
estimate. Through numerical experiments of manifold-gradient mutual
information, we show this behavior acts as a function of the effective problem
dimensionality and number of training points. On real-world datasets and
multiple zeroth-order attacks using dimension-reduction, we observe the same
universal behavior to produce samples closer to the data manifold. This results
in up to two-fold decrease in the manifold distance measure, regardless of the
model robustness. Our results suggest that taking the manifold-gradient mutual
information into account can thus inform better robust model design in the
future, and avoid leakage of the sensitive data manifold.
Related papers
- Adversarial Vulnerability as a Consequence of On-Manifold Inseparibility [16.998477658358773]
We consider classification tasks and characterize the data distribution as a low-dimensional manifold.
We argue that clean training experiences poor convergence in the off-manifold direction caused by the ill-conditioning.
We perform experiments and exhibit tremendous robustness improvements in clean training through long training and the employment of second-order methods.
arXiv Detail & Related papers (2024-10-09T14:18:52Z) - Gradient-Based Feature Learning under Structured Data [57.76552698981579]
In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction.
We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue.
In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
arXiv Detail & Related papers (2023-09-07T16:55:50Z) - On-Manifold Projected Gradient Descent [0.0]
This work provides a computable, direct, and mathematically rigorous approximation to the differential geometry of class manifold for high-dimensional data.
Tools are applied to the setting of neural network image classifiers, where we generate novel, on-manifold data samples.
arXiv Detail & Related papers (2023-08-23T17:50:50Z) - On Counterfactual Data Augmentation Under Confounding [30.76982059341284]
Counterfactual data augmentation has emerged as a method to mitigate confounding biases in the training data.
These biases arise due to various observed and unobserved confounding variables in the data generation process.
We show how our simple augmentation method helps existing state-of-the-art methods achieve good results.
arXiv Detail & Related papers (2023-05-29T16:20:23Z) - Manifold Learning by Mixture Models of VAEs for Inverse Problems [1.5749416770494704]
We learn a mixture model of variational autoencoders to represent a manifold of arbitrary topology.
We use it for solving inverse problems by minimizing a data fidelity term restricted to the learned manifold.
We demonstrate the performance of our method for low-dimensional toy examples as well as for deblurring and electrical impedance tomography.
arXiv Detail & Related papers (2023-03-27T14:29:04Z) - Convolutional Filtering on Sampled Manifolds [122.06927400759021]
We show that convolutional filtering on a sampled manifold converges to continuous manifold filtering.
Our findings are further demonstrated empirically on a problem of navigation control.
arXiv Detail & Related papers (2022-11-20T19:09:50Z) - The Manifold Hypothesis for Gradient-Based Explanations [55.01671263121624]
gradient-based explanation algorithms provide perceptually-aligned explanations.
We show that the more a feature attribution is aligned with the tangent space of the data, the more perceptually-aligned it tends to be.
We suggest that explanation algorithms should actively strive to align their explanations with the data manifold.
arXiv Detail & Related papers (2022-06-15T08:49:24Z) - Improving Diffusion Models for Inverse Problems using Manifold Constraints [55.91148172752894]
We show that current solvers throw the sample path off the data manifold, and hence the error accumulates.
To address this, we propose an additional correction term inspired by the manifold constraint.
We show that our method is superior to the previous methods both theoretically and empirically.
arXiv Detail & Related papers (2022-06-02T09:06:10Z) - Robust and On-the-fly Dataset Denoising for Image Classification [72.10311040730815]
On-the-fly Data Denoising (ODD) is robust to mislabeled examples, while introducing almost zero computational overhead compared to standard training.
ODD is able to achieve state-of-the-art results on a wide range of datasets including real-world ones such as WebVision and Clothing1M.
arXiv Detail & Related papers (2020-03-24T03:59:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.