Revealing the Distributional Vulnerability of Discriminators by Implicit
Generators
- URL: http://arxiv.org/abs/2108.09976v4
- Date: Sun, 27 Aug 2023 07:12:40 GMT
- Title: Revealing the Distributional Vulnerability of Discriminators by Implicit
Generators
- Authors: Zhilin Zhao and Longbing Cao and Kun-Yu Lin
- Abstract summary: In deep neural learning, a discriminator trained on in-distribution (ID) samples may make high-confidence predictions on out-of-distribution (OOD) samples.
We propose a general approach for itfine-tuning discriminators by implicit generators (FIG)
It improves the ability of a standard discriminator in distinguishing ID and OOD samples by generating and penalizing its specific OOD samples.
- Score: 36.66825830101456
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In deep neural learning, a discriminator trained on in-distribution (ID)
samples may make high-confidence predictions on out-of-distribution (OOD)
samples. This triggers a significant matter for robust, trustworthy and safe
deep learning. The issue is primarily caused by the limited ID samples
observable in training the discriminator when OOD samples are unavailable. We
propose a general approach for \textit{fine-tuning discriminators by implicit
generators} (FIG). FIG is grounded on information theory and applicable to
standard discriminators without retraining. It improves the ability of a
standard discriminator in distinguishing ID and OOD samples by generating and
penalizing its specific OOD samples. According to the Shannon entropy, an
energy-based implicit generator is inferred from a discriminator without extra
training costs. Then, a Langevin dynamic sampler draws specific OOD samples for
the implicit generator. Lastly, we design a regularizer fitting the design
principle of the implicit generator to induce high entropy on those generated
OOD samples. The experiments on different networks and datasets demonstrate
that FIG achieves the state-of-the-art OOD detection performance.
Related papers
- Pseudo Outlier Exposure for Out-of-Distribution Detection using
Pretrained Transformers [3.8839179829686126]
A rejection network can be trained with ID and diverse outlier samples to detect test OOD samples.
We propose a method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes.
Our method does not require any external OOD data and can be easily implemented within off-the-shelf Transformers.
arXiv Detail & Related papers (2023-07-18T17:29:23Z) - Energy-bounded Learning for Robust Models of Code [16.592638312365164]
In programming, learning code representations has a variety of applications, including code classification, code search, comment generation, bug prediction, and so on.
We propose the use of an energy-bounded learning objective function to assign a higher score to in-distribution samples and a lower score to out-of-distribution samples in order to incorporate such out-of-distribution samples into the training process of source code models.
arXiv Detail & Related papers (2021-12-20T06:28:56Z) - Approximating Probability Distributions by using Wasserstein Generative
Adversarial Networks [16.005358327268194]
Wasserstein generative adversarial networks (WGANs) with GroupSort neural networks as their discriminators are studied.
It is shown that the error bound of the approximation for the target distribution depends on the width and depth (capacity) of the generators and discriminators.
arXiv Detail & Related papers (2021-03-18T07:40:13Z) - Bridging In- and Out-of-distribution Samples for Their Better
Discriminability [18.84265231678354]
We consider samples lying in the intermediate of the two and use them for training a network.
We generate such samples using multiple image transformations that corrupt inputs in various ways and with different severity levels.
We estimate where the generated samples by a single image transformation lie between ID and OOD using a network trained on clean ID samples.
arXiv Detail & Related papers (2021-01-07T11:34:18Z) - Learn what you can't learn: Regularized Ensembles for Transductive
Out-of-distribution Detection [76.39067237772286]
We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios.
This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data.
We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
arXiv Detail & Related papers (2020-12-10T16:55:13Z) - Sampling-Decomposable Generative Adversarial Recommender [84.05894139540048]
We propose a Sampling-Decomposable Generative Adversarial Recommender (SD-GAR)
In the framework, the divergence between some generator and the optimum is compensated by self-normalized importance sampling.
We extensively evaluate the proposed algorithm with five real-world recommendation datasets.
arXiv Detail & Related papers (2020-11-02T13:19:10Z) - Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling
by Exploring Energy of the Discriminator [85.68825725223873]
Generative Adversarial Networks (GANs) have shown great promise in modeling high dimensional data.
We introduce the Discriminator Contrastive Divergence, which is well motivated by the property of WGAN's discriminator.
We demonstrate the benefits of significant improved generation on both synthetic data and several real-world image generation benchmarks.
arXiv Detail & Related papers (2020-04-05T01:50:16Z) - Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction.
We put forward an alternative measure of anomaly score to replace the reconstruction-based metric.
Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
arXiv Detail & Related papers (2020-03-24T08:26:58Z) - Self-Adversarial Learning with Comparative Discrimination for Text
Generation [111.18614166615968]
We propose a novel self-adversarial learning (SAL) paradigm for improving GANs' performance in text generation.
During training, SAL rewards the generator when its currently generated sentence is found to be better than its previously generated samples.
Experiments on text generation benchmark datasets show that our proposed approach substantially improves both the quality and the diversity.
arXiv Detail & Related papers (2020-01-31T07:50:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.