Autoencoders
- URL: http://arxiv.org/abs/2003.05991v2
- Date: Sat, 3 Apr 2021 11:18:12 GMT
- Title: Autoencoders
- Authors: Dor Bank, Noam Koenigstein, Raja Giryes
- Abstract summary: An autoencoder is a specific type of a neural network, which is mainly designed to encode the input into a compressed and meaningful representation, and then decode it back such that the reconstructed input is similar as possible to the original one.
This chapter surveys the different types of autoencoders that are mainly used today.
- Score: 43.991924654575975
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: An autoencoder is a specific type of a neural network, which is mainly
designed to encode the input into a compressed and meaningful representation,
and then decode it back such that the reconstructed input is similar as
possible to the original one. This chapter surveys the different types of
autoencoders that are mainly used today. It also describes various applications
and use-cases of autoencoders.
Related papers
- Triple-Encoders: Representations That Fire Together, Wire Together [51.15206713482718]
Contrastive Learning is a representation learning method that encodes relative distances between utterances into the embedding space via a bi-encoder.
This study introduces triple-encoders, which efficiently compute distributed utterance mixtures from these independently encoded utterances.
We find that triple-encoders lead to a substantial improvement over bi-encoders, and even to better zero-shot generalization than single-vector representation models.
arXiv Detail & Related papers (2024-02-19T18:06:02Z) - Decoder-Only or Encoder-Decoder? Interpreting Language Model as a
Regularized Encoder-Decoder [75.03283861464365]
The seq2seq task aims at generating the target sequence based on the given input source sequence.
Traditionally, most of the seq2seq task is resolved by an encoder to encode the source sequence and a decoder to generate the target text.
Recently, a bunch of new approaches have emerged that apply decoder-only language models directly to the seq2seq task.
arXiv Detail & Related papers (2023-04-08T15:44:29Z) - On a Mechanism Framework of Autoencoders [0.0]
This paper proposes a theoretical framework on the mechanism of autoencoders.
Results of ReLU autoencoders are generalized to some non-ReLU cases.
Compared to PCA and decision trees, the advantages of (generalized) autoencoders on dimensionality reduction and classification are demonstrated.
arXiv Detail & Related papers (2022-08-15T03:51:40Z) - Diffsound: Discrete Diffusion Model for Text-to-sound Generation [78.4128796899781]
We propose a novel text-to-sound generation framework that consists of a text encoder, a Vector Quantized Variational Autoencoder (VQ-VAE), a decoder, and a vocoder.
The framework first uses the decoder to transfer the text features extracted from the text encoder to a mel-spectrogram with the help of VQ-VAE, and then the vocoder is used to transform the generated mel-spectrogram into a waveform.
arXiv Detail & Related papers (2022-07-20T15:41:47Z) - An Introduction to Autoencoders [0.0]
This article covers the mathematics and the fundamental concepts of autoencoders.
We will start with a general introduction to autoencoders, and we will discuss the role of the activation function in the output layer and the loss function.
arXiv Detail & Related papers (2022-01-11T11:55:32Z) - Bag-of-Vectors Autoencoders for Unsupervised Conditional Text Generation [18.59238482225795]
We extend Mai et al.'s proposed Emb2Emb method to learn mappings in the embedding space of an autoencoder.
We propose Bag-of-AEs Autoencoders (BoV-AEs), which encode the text into a variable-size bag of vectors that grows with the size of the text.
This allows to encode and reconstruct much longer texts than standard autoencoders.
arXiv Detail & Related papers (2021-10-13T19:30:40Z) - DeepA: A Deep Neural Analyzer For Speech And Singing Vocoding [71.73405116189531]
We propose a neural vocoder that extracts F0 and timbre/aperiodicity encoding from the input speech that emulates those defined in conventional vocoders.
As the deep neural analyzer is learnable, it is expected to be more accurate for signal reconstruction and manipulation, and generalizable from speech to singing.
arXiv Detail & Related papers (2021-10-13T01:39:57Z) - Dynamic Neural Representational Decoders for High-Resolution Semantic
Segmentation [98.05643473345474]
We propose a novel decoder, termed dynamic neural representational decoder (NRD)
As each location on the encoder's output corresponds to a local patch of the semantic labels, in this work, we represent these local patches of labels with compact neural networks.
This neural representation enables our decoder to leverage the smoothness prior in the semantic label space, and thus makes our decoder more efficient.
arXiv Detail & Related papers (2021-07-30T04:50:56Z) - Cascade Decoders-Based Autoencoders for Image Reconstruction [2.924868086534434]
This paper aims for image reconstruction of autoencoders, employs cascade decoders-based autoencoders.
The proposed serial decoders-based autoencoders include the architectures of multi-level decoders and the related optimization algorithms.
It is evaluated by the experimental results that the proposed autoencoders outperform the classical autoencoders in the performance of image reconstruction.
arXiv Detail & Related papers (2021-06-29T23:40:54Z) - A Showcase of the Use of Autoencoders in Feature Learning Applications [11.329636084818778]
Autoencoders are techniques for data representation learning based on artificial neural networks.
This work presents these applications and provides details on how autoencoders can perform them, including code samples making use of an R package with an easy-to-use interface for autoencoder design and training.
arXiv Detail & Related papers (2020-05-08T23:56:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.