Gradient-Based Adversarial and Out-of-Distribution Detection
- URL: http://arxiv.org/abs/2206.08255v1
- Date: Thu, 16 Jun 2022 15:50:41 GMT
- Title: Gradient-Based Adversarial and Out-of-Distribution Detection
- Authors: Jinsol Lee, Mohit Prabhushankar, Ghassan AlRegib
- Abstract summary: We introduce confounding labels in gradient generation to probe the effective expressivity of neural networks.
We show that our gradient-based approach allows for capturing the anomaly in inputs based on the effective expressivity of the models.
- Score: 15.510581400494207
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose to utilize gradients for detecting adversarial and
out-of-distribution samples. We introduce confounding labels -- labels that
differ from normal labels seen during training -- in gradient generation to
probe the effective expressivity of neural networks. Gradients depict the
amount of change required for a model to properly represent given inputs,
providing insight into the representational power of the model established by
network architectural properties as well as training data. By introducing a
label of different design, we remove the dependency on ground truth labels for
gradient generation during inference. We show that our gradient-based approach
allows for capturing the anomaly in inputs based on the effective expressivity
of the models with no hyperparameter tuning or additional processing, and
outperforms state-of-the-art methods for adversarial and out-of-distribution
detection.
Related papers
- Derivative-Free Diffusion Manifold-Constrained Gradient for Unified XAI [59.96044730204345]
We introduce Derivative-Free Diffusion Manifold-Constrainted Gradients (FreeMCG)
FreeMCG serves as an improved basis for explainability of a given neural network.
We show that our method yields state-of-the-art results while preserving the essential properties expected of XAI tools.
arXiv Detail & Related papers (2024-11-22T11:15:14Z) - Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels.
Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
arXiv Detail & Related papers (2024-04-26T06:00:27Z) - Fascinating Supervisory Signals and Where to Find Them: Deep Anomaly
Detection with Scale Learning [11.245813423781415]
We devise novel data-driven supervision for data by introducing a characteristic -- scale -- as data labels.
Scales serve as labels attached to transformed representations, thus offering ample labeled data for neural network training.
This paper further proposes a scale learning-based anomaly detection method.
arXiv Detail & Related papers (2023-05-25T14:48:00Z) - Reflected Diffusion Models [93.26107023470979]
We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data.
Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
arXiv Detail & Related papers (2023-04-10T17:54:38Z) - Probing the Purview of Neural Networks via Gradient Analysis [13.800680101300756]
We analyze the data-dependent capacity of neural networks and assess anomalies in inputs from the perspective of networks during inference.
To probe the purview of a network, we utilize gradients to measure the amount of change required for the model to characterize the given inputs more accurately.
We demonstrate that our gradient-based approach can effectively differentiate inputs that cannot be accurately represented with learned features.
arXiv Detail & Related papers (2023-04-06T03:02:05Z) - ContraFeat: Contrasting Deep Features for Semantic Discovery [102.4163768995288]
StyleGAN has shown strong potential for disentangled semantic control.
Existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results.
We propose a model that automates this process and achieves state-of-the-art semantic discovery performance.
arXiv Detail & Related papers (2022-12-14T15:22:13Z) - Interpretation of Neural Networks is Susceptible to Universal Adversarial Perturbations [9.054540533394926]
We show the existence of a Universal Perturbation for Interpretation (UPI) for standard image datasets.
We propose a gradient-based optimization method as well as a principal component analysis (PCA)-based approach to compute a UPI which can effectively alter a neural network's gradient-based interpretation on different samples.
arXiv Detail & Related papers (2022-11-30T15:55:40Z) - Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold.
We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples.
We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
arXiv Detail & Related papers (2022-06-23T14:16:30Z) - Bayesian Graph Contrastive Learning [55.36652660268726]
We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders.
Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector.
We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
arXiv Detail & Related papers (2021-12-15T01:45:32Z) - Revealing and Protecting Labels in Distributed Training [3.18475216176047]
We propose a method to discover the set of labels of training samples from only the gradient of the last layer and the id to label mapping.
We demonstrate the effectiveness of our method for model training in two domains - image classification, and automatic speech recognition.
arXiv Detail & Related papers (2021-10-31T17:57:49Z) - Backpropagated Gradient Representations for Anomaly Detection [19.191613437266184]
Anomalies require more drastic model updates to fully represent them compared to normal data.
We show that the proposed method using gradient-based representations achieves state-of-the-art anomaly detection performance in benchmark image recognition datasets.
arXiv Detail & Related papers (2020-07-18T19:39:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.