Robust Perception through Equivariance

• URL: http://arxiv.org/abs/2212.06079v2
• Date: Sun, 4 Jun 2023 01:02:57 GMT
• Title: Robust Perception through Equivariance
• Authors: Chengzhi Mao, Lingyu Zhang, Abhishek Joshi, Junfeng Yang, Hao Wang, Carl Vondrick
• Abstract summary: We introduce a framework that uses the dense intrinsic constraints in natural images to robustify inference. By introducing constraints at inference time, we can shift the burden of robustness from training to the inference algorithm.
• Score: 28.43219868475906
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep networks for computer vision are not reliable when they encounter adversarial examples. In this paper, we introduce a framework that uses the dense intrinsic constraints in natural images to robustify inference. By introducing constraints at inference time, we can shift the burden of robustness from training to the inference algorithm, thereby allowing the model to adjust dynamically to each individual image's unique and potentially novel characteristics at inference time. Among different constraints, we find that equivariance-based constraints are most effective, because they allow dense constraints in the feature space without overly constraining the representation at a fine-grained level. Our theoretical results validate the importance of having such dense constraints at inference time. Our empirical experiments show that restoring feature equivariance at inference time defends against worst-case adversarial perturbations. The method obtains improved adversarial robustness on four datasets (ImageNet, Cityscapes, PASCAL VOC, and MS-COCO) on image recognition, semantic segmentation, and instance segmentation tasks. Project page is available at equi4robust.cs.columbia.edu.

Related papers

• Counterfactual Image Generation for adversarially robust and interpretable Classifiers [1.3859669037499769]
  We propose a unified framework leveraging image-to-image translation Generative Adrial Networks (GANs) to produce counterfactual samples. This is achieved by combining the classifier and discriminator into a single model that attributes real images to their respective classes and flags generated images as "fake" We show how the model exhibits improved robustness to adversarial attacks, and we show how the discriminator's "fakeness" value serves as an uncertainty measure of the predictions.
  arXiv  Detail & Related papers  (2023-10-01T18:50:29Z)
• Towards Practical Control of Singular Values of Convolutional Layers [65.25070864775793]
  Convolutional neural networks (CNNs) are easy to train, but their essential properties, such as generalization error and adversarial robustness, are hard to control. Recent research demonstrated that singular values of convolutional layers significantly affect such elusive properties. We offer a principled approach to alleviating constraints of the prior art at the expense of an insignificant reduction in layer expressivity.
  arXiv  Detail & Related papers  (2022-11-24T19:09:44Z)
• Robustness and invariance properties of image classifiers [8.970032486260695]
  Deep neural networks have achieved impressive results in many image classification tasks. Deep networks are not robust to a large variety of semantic-preserving image modifications. The poor robustness of image classifiers to small data distribution shifts raises serious concerns regarding their trustworthiness.
  arXiv  Detail & Related papers  (2022-08-30T11:00:59Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• Adversarial Robustness Across Representation Spaces [35.58913661509278]
  Adversa robustness corresponds to the susceptibility of deep neural networks to imperceptible perturbations made at test time. In this work we extend the setting to consider the problem of training of deep neural networks that can be made simultaneously robust to perturbations applied in multiple natural representation spaces.
  arXiv  Detail & Related papers  (2020-12-01T19:55:58Z)
• Adversarial Robustness of Supervised Sparse Coding [34.94566482399662]
  We consider a model that involves learning a representation while at the same time giving a precise generalization bound and a robustness certificate. We focus on the hypothesis class obtained by combining a sparsity-promoting encoder coupled with a linear encoder. We provide a robustness certificate for end-to-end classification.
  arXiv  Detail & Related papers  (2020-10-22T22:05:21Z)
• Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders [63.46738617561255]
  We consider the problem of sparsifying the discrete latent space of a trained conditional variational autoencoder. We use evidential theory to identify the latent classes that receive direct evidence from a particular input condition and filter out those that do not. Experiments on diverse tasks, such as image generation and human behavior prediction, demonstrate the effectiveness of our proposed technique.
  arXiv  Detail & Related papers  (2020-10-19T01:27:21Z)
• Learning perturbation sets for robust machine learning [97.6757418136662]
  We use a conditional generator that defines the perturbation set over a constrained region of the latent space. We measure the quality of our learned perturbation sets both quantitatively and qualitatively. We leverage our learned perturbation sets to train models which are empirically and certifiably robust to adversarial image corruptions and adversarial lighting variations.
  arXiv  Detail & Related papers  (2020-07-16T16:39:54Z)
• Towards Certified Robustness of Distance Metric Learning [53.96113074344632]
  We advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms. We show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness.
  arXiv  Detail & Related papers  (2020-06-10T16:51:53Z)
• Learning to Manipulate Individual Objects in an Image [71.55005356240761]
  We describe a method to train a generative model with latent factors that are independent and localized. This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects. Unlike other unsupervised generative models, ours enables object-centric manipulation, without requiring object-level annotations.
  arXiv  Detail & Related papers  (2020-04-11T21:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.