Robustifying Deep Vision Models Through Shape Sensitization

• URL: http://arxiv.org/abs/2211.07277v1
• Date: Mon, 14 Nov 2022 11:17:46 GMT
• Title: Robustifying Deep Vision Models Through Shape Sensitization
• Authors: Aditay Tripathi, Rishubh Singh, Anirban Chakraborty, Pradeep Shenoy
• Abstract summary: We propose a simple, lightweight adversarial augmentation technique that explicitly incentivizes the network to learn holistic shapes. Our augmentations superpose edgemaps from one image onto another image with shuffled patches, using a randomly determined mixing proportion. We show that our augmentations significantly improve classification accuracy and robustness measures on a range of datasets and neural architectures.
• Score: 19.118696557797957
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work has shown that deep vision models tend to be overly dependent on low-level or "texture" features, leading to poor generalization. Various data augmentation strategies have been proposed to overcome this so-called texture bias in DNNs. We propose a simple, lightweight adversarial augmentation technique that explicitly incentivizes the network to learn holistic shapes for accurate prediction in an object classification setting. Our augmentations superpose edgemaps from one image onto another image with shuffled patches, using a randomly determined mixing proportion, with the image label of the edgemap image. To classify these augmented images, the model needs to not only detect and focus on edges but distinguish between relevant and spurious edges. We show that our augmentations significantly improve classification accuracy and robustness measures on a range of datasets and neural architectures. As an example, for ViT-S, We obtain absolute gains on classification accuracy gains up to 6%. We also obtain gains of up to 28% and 8.5% on natural adversarial and out-of-distribution datasets like ImageNet-A (for ViT-B) and ImageNet-R (for ViT-S), respectively. Analysis using a range of probe datasets shows substantially increased shape sensitivity in our trained models, explaining the observed improvement in robustness and classification accuracy.

Related papers

• Geometric Data Augmentations to Mitigate Distribution Shifts in Pollen Classification from Microscopic Images [4.545340728210854]
  We leverage the domain knowledge that geometric features are highly important for accurate pollen identification. We introduce two novel geometric image augmentation techniques to significantly narrow the accuracy gap between the model performance on the train and test datasets.
  arXiv  Detail & Related papers  (2023-11-18T10:35:18Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification [9.779748872936912]
  We show that proper framing of the input image can lead to the correct classification of 98.91% of ImageNet images. We propose a test-time augmentation (TTA) technique that improves classification accuracy by forcing models to explicitly perform zoom-in operations.
  arXiv  Detail & Related papers  (2023-04-11T23:55:50Z)
• DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator [53.57431705309919]
  ImageNet pre-trained deep neural networks (DNNs) show notable transferability for building effective image quality assessment (IQA) models. We develop a novel full-reference IQA (FR-IQA) model based exclusively on pre-trained DNN features. We conduct comprehensive experiments to demonstrate the superiority of the proposed quality model on five standard IQA datasets.
  arXiv  Detail & Related papers  (2022-11-09T14:57:27Z)
• Multi-layer Representation Learning for Robust OOD Image Classification [3.1372269816123994]
  We argue that extracting features from a CNN's intermediate layers can assist in the model's final prediction. Specifically, we adapt the Hypercolumns method to a ResNet-18 and find a significant increase in the model's accuracy, when evaluating on the NICO dataset.
  arXiv  Detail & Related papers  (2022-07-27T17:46:06Z)
• A Comprehensive Study of Image Classification Model Sensitivity to Foregrounds, Backgrounds, and Visual Attributes [58.633364000258645]
  We call this dataset RIVAL10 consisting of roughly $26k$ instances over $10$ classes. We evaluate the sensitivity of a broad set of models to noise corruptions in foregrounds, backgrounds and attributes. In our analysis, we consider diverse state-of-the-art architectures (ResNets, Transformers) and training procedures (CLIP, SimCLR, DeiT, Adversarial Training)
  arXiv  Detail & Related papers  (2022-01-26T06:31:28Z)
• Towards Robustness of Neural Networks [0.0]
  We introduce ImageNet-A/O and ImageNet-R as well as a synthetic environment and testing suite we called CAOS. All of the datasets were created for testing robustness and measuring progress in robustness. We build off of simple baselines in the form of Maximum Logit, and Typicality Score as well as create a novel data augmentation method in the form of DeepAugment.
  arXiv  Detail & Related papers  (2021-12-30T19:41:10Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• Contemplating real-world object classification [53.10151901863263]
  We reanalyze the ObjectNet dataset recently proposed by Barbu et al. containing objects in daily life situations. We find that applying deep models to the isolated objects, rather than the entire scene as is done in the original paper, results in around 20-30% performance improvement.
  arXiv  Detail & Related papers  (2021-03-08T23:29:59Z)
• Robust Data Hiding Using Inverse Gradient Attention [82.73143630466629]
  In the data hiding task, each pixel of cover images should be treated differently since they have divergent tolerabilities. We propose a novel deep data hiding scheme with Inverse Gradient Attention (IGA), combing the ideas of adversarial learning and attention mechanism. Empirically, extensive experiments show that the proposed model outperforms the state-of-the-art methods on two prevalent datasets.
  arXiv  Detail & Related papers  (2020-11-21T19:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.