Super-resolution Variational Auto-Encoders

• URL: http://arxiv.org/abs/2006.05218v2
• Date: Tue, 30 Jun 2020 13:06:43 GMT
• Title: Super-resolution Variational Auto-Encoders
• Authors: Ioannis Gatopoulos, Maarten Stol, Jakub M. Tomczak
• Abstract summary: We propose to enhance VAEs by adding a random variable that is a downscaled version of the original image. We present empirically that the proposed approach performs comparably to VAEs in terms of the negative log-likelihood.
• Score: 8.873449722727026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The framework of variational autoencoders (VAEs) provides a principled method for jointly learning latent-variable models and corresponding inference models. However, the main drawback of this approach is the blurriness of the generated images. Some studies link this effect to the objective function, namely, the (negative) log-likelihood. Here, we propose to enhance VAEs by adding a random variable that is a downscaled version of the original image and still use the log-likelihood function as the learning objective. Further, by providing the downscaled image as an input to the decoder, it can be used in a manner similar to the super-resolution. We present empirically that the proposed approach performs comparably to VAEs in terms of the negative log-likelihood, but it obtains a better FID score in data synthesis.

Related papers

• Image Harmonization using Robust Restricted CDF Matching [0.0]
  We present a method of image harmonization using Cumulative Distribution Function (CDF) matching based on curve fitting. The transformation of image intensities is non-linear but still smooth and elastic", as compared to other known histogram matching algorithms. Even though we demonstrate our method for MRI images, the method is generic enough to apply to other types of imaging data.
  arXiv  Detail & Related papers  (2024-11-20T12:12:47Z)
• CR-VAE: Contrastive Regularization on Variational Autoencoders for Preventing Posterior Collapse [1.0044057719679085]
  The Variational Autoencoder (VAE) is known to suffer from the phenomenon of textitposterior collapse We propose a novel solution, the Contrastive Regularization for Variational Autoencoders (CR-VAE)
  arXiv  Detail & Related papers  (2023-09-06T13:05:42Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• Lossy Image Compression with Conditional Diffusion Models [25.158390422252097]
  This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models. In contrast to VAE-based neural compression, where the (mean) decoder is a deterministic neural network, our decoder is a conditional diffusion model. Our approach yields stronger reported FID scores than the GAN-based model, while also yielding competitive performance with VAE-based models in several distortion metrics.
  arXiv  Detail & Related papers  (2022-09-14T21:53:27Z)
• CCLF: A Contrastive-Curiosity-Driven Learning Framework for Sample-Efficient Reinforcement Learning [56.20123080771364]
  We develop a model-agnostic Contrastive-Curiosity-Driven Learning Framework (CCLF) for reinforcement learning. CCLF fully exploit sample importance and improve learning efficiency in a self-supervised manner. We evaluate this approach on the DeepMind Control Suite, Atari, and MiniGrid benchmarks.
  arXiv  Detail & Related papers  (2022-05-02T14:42:05Z)
• Neural Data-Dependent Transform for Learned Image Compression [72.86505042102155]
  We build a neural data-dependent transform and introduce a continuous online mode decision mechanism to jointly optimize the coding efficiency for each individual image. The experimental results show the effectiveness of the proposed neural-syntax design and the continuous online mode decision mechanism.
  arXiv  Detail & Related papers  (2022-03-09T14:56:48Z)
• Diffusion-Based Representation Learning [65.55681678004038]
  We augment the denoising score matching framework to enable representation learning without any supervised signal. In contrast, the introduced diffusion-based representation learning relies on a new formulation of the denoising score matching objective. Using the same approach, we propose to learn an infinite-dimensional latent code that achieves improvements of state-of-the-art models on semi-supervised image classification.
  arXiv  Detail & Related papers  (2021-05-29T09:26:02Z)
• Anomaly detection through latent space restoration using vector-quantized variational autoencoders [0.8122270502556374]
  We propose an out-of-distribution detection method using density and restoration-based approaches. The VQ-VAE model learns to encode images in a categorical latent space. The prior distribution of latent codes is then modelled using an Auto-Regressive (AR) model.
  arXiv  Detail & Related papers  (2020-12-12T09:19:59Z)
• Autoencoding Variational Autoencoder [56.05008520271406]
  We study the implications of this behaviour on the learned representations and also the consequences of fixing it by introducing a notion of self consistency. We show that encoders trained with our self-consistency approach lead to representations that are robust (insensitive) to perturbations in the input introduced by adversarial attacks.
  arXiv  Detail & Related papers  (2020-12-07T14:16:14Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.