Are We Using Autoencoders in a Wrong Way?

• URL: http://arxiv.org/abs/2309.01532v1
• Date: Mon, 4 Sep 2023 11:22:43 GMT
• Title: Are We Using Autoencoders in a Wrong Way?
• Authors: Gabriele Martino, Davide Moroni, Massimo Martinelli
• Abstract summary: Autoencoders are used for dimensionality reduction, anomaly detection and feature extraction. We revisited the standard training for the undercomplete Autoencoder modifying the shape of the latent space. We also explored the behaviour of the latent space in the case of reconstruction of a random sample from the whole dataset.
• Score: 3.110260251019273
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autoencoders are certainly among the most studied and used Deep Learning models: the idea behind them is to train a model in order to reconstruct the same input data. The peculiarity of these models is to compress the information through a bottleneck, creating what is called Latent Space. Autoencoders are generally used for dimensionality reduction, anomaly detection and feature extraction. These models have been extensively studied and updated, given their high simplicity and power. Examples are (i) the Denoising Autoencoder, where the model is trained to reconstruct an image from a noisy one; (ii) Sparse Autoencoder, where the bottleneck is created by a regularization term in the loss function; (iii) Variational Autoencoder, where the latent space is used to generate new consistent data. In this article, we revisited the standard training for the undercomplete Autoencoder modifying the shape of the latent space without using any explicit regularization term in the loss function. We forced the model to reconstruct not the same observation in input, but another one sampled from the same class distribution. We also explored the behaviour of the latent space in the case of reconstruction of a random sample from the whole dataset.

Related papers

• Generative Autoencoding of Dropout Patterns [11.965844936801801]
  We propose a generative model termed Deciphering Autoencoders. We assign a unique random dropout pattern to each data point in the training dataset. We then train an autoencoder to reconstruct the corresponding data point using this pattern as information to be encoded.
  arXiv  Detail & Related papers  (2023-10-03T00:54:13Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Variational Diffusion Auto-encoder: Latent Space Extraction from Pre-trained Diffusion Models [0.0]
  Variational Auto-Encoders (VAEs) face challenges with the quality of generated images, often presenting noticeable blurriness. This issue stems from the unrealistic assumption that approximates the conditional data distribution, $p(textbfx | textbfz)$, as an isotropic Gaussian. We illustrate how one can extract a latent space from a pre-existing diffusion model by optimizing an encoder to maximize the marginal data log-likelihood.
  arXiv  Detail & Related papers  (2023-04-24T14:44:47Z)
• Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs [16.968490007064872]
  Applying variational autoencoders (VAEs) to sequential data offers a method for controlled sequence generation, manipulation, and structured representation learning. We show theoretically that this removes pointwise mutual information provided by the decoder input, which is compensated for by utilizing the latent space. Compared to uniform dropout on standard text benchmark datasets, our targeted approach increases both sequence performance and the information captured in the latent space.
  arXiv  Detail & Related papers  (2022-09-26T11:21:19Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Anomaly Detection with Prototype-Guided Discriminative Latent Embeddings [29.93983580779689]
  We present a novel approach for anomaly detection, which utilizes discriminative prototypes of normal data to reconstruct video frames. In this way, the model will favor the reconstruction of normal events and distort the reconstruction of abnormal events. We evaluate the effectiveness of our method on three benchmark datasets and experimental results demonstrate the proposed method outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2021-04-30T12:16:52Z)
• Anytime Sampling for Autoregressive Models via Ordered Autoencoding [88.01906682843618]
  Autoregressive models are widely used for tasks such as image and audio generation. The sampling process of these models does not allow interruptions and cannot adapt to real-time computational resources. We propose a new family of autoregressive models that enables anytime sampling.
  arXiv  Detail & Related papers  (2021-02-23T05:13:16Z)
• Autoencoding Variational Autoencoder [56.05008520271406]
  We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. We show that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks.
  arXiv  Detail & Related papers  (2020-12-07T14:16:14Z)
• Automatic Recall Machines: Internal Replay, Continual Learning and the Brain [104.38824285741248]
  Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective. Instead the implicit memory of learned samples within the assessed model itself is exploited.
  arXiv  Detail & Related papers  (2020-06-22T15:07:06Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)
• Iterative energy-based projection on a normal data manifold for anomaly localization [3.785123406103385]
  We propose a new approach for projecting anomalous data on a autoencoder-learned normal data manifold. By iteratively updating the input of the autoencoder, we bypass the loss of high-frequency information caused by the autoencoder bottleneck.
  arXiv  Detail & Related papers  (2020-02-10T13:35:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.