In-distribution adversarial attacks on object recognition models using gradient-free search

• URL: http://arxiv.org/abs/2106.16198v3
• Date: Thu, 30 Jan 2025 19:58:53 GMT
• Title: In-distribution adversarial attacks on object recognition models using gradient-free search
• Authors: Spandan Madan, Tomotake Sasaki, Hanspeter Pfister, Tzu-Mao Li, Xavier Boix,
• Abstract summary: We present evidence of perturbed images within the training data distribution, which networks fail to classify.<n>We train models on data sampled from parametric distributions, then search inside this data distribution to find such in-distribution adversarial examples.<n>Findings also extend to natural images from ImageNet and Co3D datasets.
• Score: 27.17685074149947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks are susceptible to small perturbations in the form of 2D rotations and shifts, image crops, and even changes in object colors. Past works attribute these errors to dataset bias, claiming that models fail on these perturbed samples as they do not belong to the training data distribution. Here, we challenge this claim and present evidence of the widespread existence of perturbed images within the training data distribution, which networks fail to classify. We train models on data sampled from parametric distributions, then search inside this data distribution to find such in-distribution adversarial examples. This is done using our gradient-free evolution strategies (ES) based approach which we call CMA-Search. Despite training with a large-scale (0.5 million images), unbiased dataset of camera and light variations, CMA-Search can find a failure inside the data distribution in over 71% cases by perturbing the camera position. With lighting changes, CMA-Search finds misclassifications in 42% cases. These findings also extend to natural images from ImageNet and Co3D datasets. This phenomenon of in-distribution images presents a highly worrisome problem for artificial intelligence -- they bypass the need for a malicious agent to add engineered noise to induce an adversarial attack. All code, datasets, and demos are available at https://github.com/Spandan-Madan/in_distribution_adversarial_examples.

Related papers

• Bayesian generative models can flag performance loss, bias, and out-of-distribution image content [15.835055687646507]
  Generative models are popular for medical imaging tasks such as anomaly detection, feature extraction, data visualization, or image generation. Since they are parameterized by deep learning models, they are often sensitive to distribution shifts and unreliable when applied to out-of-distribution data. We show how pixel-wise uncertainty can detect out-of-distribution image content such as ink, rulers, and patches.
  arXiv  Detail & Related papers  (2025-03-21T18:45:28Z)
• Leveraging Text-to-Image Generation for Handling Spurious Correlation [24.940576844328408]
  Deep neural networks trained with Empirical Risk Minimization (ERM) perform well when both training and test data come from the same domain. ERM models may rely on spurious correlations that often exist between labels and irrelevant features of images, making predictions unreliable when those features do not exist. We propose a technique to generate training samples with text-to-image (T2I) diffusion models for addressing the spurious correlation problem.
  arXiv  Detail & Related papers  (2025-03-21T15:28:22Z)
• Feature Corrective Transfer Learning: End-to-End Solutions to Object Detection in Non-Ideal Visual Conditions [11.90136900277127]
  "Feature Corrective Transfer Learning" is a novel approach to facilitate the end-to-end detection of objects in challenging scenarios. Non-ideal images are processed by comparing their feature maps against those from the initial ideal RGB model. This approach refines the model's ability to perform object detection across varying conditions through direct feature map correction.
  arXiv  Detail & Related papers  (2024-04-17T09:58:53Z)
• The Journey, Not the Destination: How Data Guides Diffusion Models [75.19694584942623]
  Diffusion models trained on large datasets can synthesize photo-realistic images of remarkable quality and diversity. We propose a framework that: (i) provides a formal notion of data attribution in the context of diffusion models, and (ii) allows us to counterfactually validate such attributions.
  arXiv  Detail & Related papers  (2023-12-11T08:39:43Z)
• Learning Defect Prediction from Unrealistic Data [57.53586547895278]
  Pretrained models of code have become popular choices for code understanding and generation tasks. Such models tend to be large and require commensurate volumes of training data. It has become popular to train models with far larger but less realistic datasets, such as functions with artificially injected bugs. Models trained on such data tend to only perform well on similar data, while underperforming on real world programs.
  arXiv  Detail & Related papers  (2023-11-02T01:51:43Z)
• Revisiting Data Augmentation for Rotational Invariance in Convolutional Neural Networks [0.29127054707887967]
  We investigate how best to include rotational invariance in a CNN for image classification. Our experiments show that networks trained with data augmentation alone can classify rotated images nearly as well as in the normal unrotated case.
  arXiv  Detail & Related papers  (2023-10-12T15:53:24Z)
• On quantifying and improving realism of images generated with diffusion [50.37578424163951]
  We propose a metric, called Image Realism Score (IRS), computed from five statistical measures of a given image. IRS is easily usable as a measure to classify a given image as real or fake. We experimentally establish the model- and data-agnostic nature of the proposed IRS by successfully detecting fake images generated by Stable Diffusion Model (SDM), Dalle2, Midjourney and BigGAN. Our efforts have also led to Gen-100 dataset, which provides 1,000 samples for 100 classes generated by four high-quality models.
  arXiv  Detail & Related papers  (2023-09-26T08:32:55Z)
• The Change You Want to See (Now in 3D) [65.61789642291636]
  The goal of this paper is to detect what has changed, if anything, between two "in the wild" images of the same 3D scene. We contribute a change detection model that is trained entirely on synthetic data and is class-agnostic. We release a new evaluation dataset consisting of real-world image pairs with human-annotated differences.
  arXiv  Detail & Related papers  (2023-08-21T01:59:45Z)
• Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision [76.32860119056964]
  We propose a novel class of denoising diffusion probabilistic models that learn to sample from distributions of signals that are never directly observed. We demonstrate the effectiveness of our method on three challenging computer vision tasks.
  arXiv  Detail & Related papers  (2023-06-20T17:53:00Z)
• GSURE-Based Diffusion Model Training with Corrupted Data [35.56267114494076]
  We propose a novel training technique for generative diffusion models based only on corrupted data. We demonstrate our technique on face images as well as Magnetic Resonance Imaging (MRI)
  arXiv  Detail & Related papers  (2023-05-22T15:27:20Z)
• Scale-Equivariant Deep Learning for 3D Data [44.52688267348063]
  Convolutional neural networks (CNNs) recognize objects regardless of their position in the image. We propose a scale-equivariant convolutional network layer for three-dimensional data. Our experiments demonstrate the effectiveness of the proposed method in achieving scale-equivariant for 3D medical image analysis.
  arXiv  Detail & Related papers  (2023-04-12T13:56:12Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• DC4L: Distribution Shift Recovery via Data-Driven Control for Deep Learning Models [4.374569172244273]
  We propose to use control for learned models to recover from distribution shifts online. Our method applies a sequence of semantic-preserving transformations to bring the shifted data closer in distribution to the training set. We show that our method generalizes to composites of shifts from the ImageNet-C benchmark, achieving improvements in average accuracy of up to 9.81%.
  arXiv  Detail & Related papers  (2023-02-20T22:06:26Z)
• Learning Transformations To Reduce the Geometric Shift in Object Detection [60.20931827772482]
  We tackle geometric shifts emerging from variations in the image capture process. We introduce a self-training approach that learns a set of geometric transformations to minimize these shifts. We evaluate our method on two different shifts, i.e., a camera's field of view (FoV) change and a viewpoint change.
  arXiv  Detail & Related papers  (2023-01-13T11:55:30Z)
• Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models [1.6352599467675781]
  We propose a method based on diffusion models to detect and segment anomalies in brain imaging. Our diffusion models achieve competitive performance compared with autoregressive approaches across a series of experiments with 2D CT and MRI data.
  arXiv  Detail & Related papers  (2022-06-07T17:30:43Z)
• Uncertainty-Aware Camera Pose Estimation from Points and Lines [101.03675842534415]
  Perspective-n-Point-and-Line (Pn$PL) aims at fast, accurate and robust camera localizations with respect to a 3D model from 2D-3D feature coordinates.
  arXiv  Detail & Related papers  (2021-07-08T15:19:36Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• Anomaly Detection in Image Datasets Using Convolutional Neural Networks, Center Loss, and Mahalanobis Distance [0.0]
  User activities generate a significant number of poor-quality or irrelevant images and data vectors. For neural networks, the anomalous is usually defined as out-of-distribution samples. This work proposes methods for supervised and semi-supervised detection of out-of-distribution samples in image datasets.
  arXiv  Detail & Related papers  (2021-04-13T13:44:03Z)
• Exploiting Invariance in Training Deep Neural Networks [4.169130102668252]
  Inspired by two basic mechanisms in animal visual systems, we introduce a feature transform technique that imposes invariance properties in the training of deep neural networks. The resulting algorithm requires less parameter tuning, trains well with an initial learning rate 1.0, and easily generalizes to different tasks. Tested on ImageNet, MS COCO, and Cityscapes datasets, our proposed technique requires fewer iterations to train, surpasses all baselines by a large margin, seamlessly works on both small and large batch size training, and applies to different computer vision tasks of image classification, object detection, and semantic segmentation.
  arXiv  Detail & Related papers  (2021-03-30T19:18:31Z)
• What Does CNN Shift Invariance Look Like? A Visualization Study [87.79405274610681]
  Feature extraction with convolutional neural networks (CNNs) is a popular method to represent images for machine learning tasks. We focus on measuring and visualizing the shift invariance of extracted features from popular off-the-shelf CNN models. We conclude that features extracted from popular networks are not globally invariant, and that biases and artifacts exist within this variance.
  arXiv  Detail & Related papers  (2020-11-09T01:16:30Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.