How Neural Losses Shape VAE Latents

• URL: http://arxiv.org/abs/2606.00635v1
• Date: Sat, 30 May 2026 09:20:56 GMT
• Title: How Neural Losses Shape VAE Latents
• Abstract summary: We show that the choice of reconstruction loss reshapes the rate-distortion problem itself.<n>We prove and verify that augmenting pointwise reconstruction with neural terms, such as perceptual and adversarial objectives, reduces the amount of information stored in the latent representations.
• Score: 15.636976191219448
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern VAEs are rarely trained with the pointwise likelihood implied by the standard $β$-VAE objective. In practice, pointwise reconstruction is often combined with perceptual and adversarial losses, despite a lack of understanding of how this changes the latent dynamics of the model. We show that the choice of reconstruction loss reshapes the rate-distortion problem itself, altering both the information content and the geometry of the learned latent space in ways that may be invisible from reconstructions alone. First, we prove and verify empirically that augmenting pointwise reconstruction with neural terms, such as perceptual and adversarial objectives, reduces the amount of information stored in the latent representations. Second, we show that neural reconstruction losses systematically change the geometry of the latent space: they make representations more isotropic and distribute uncertainty more evenly across latent dimensions, producing different posterior variance profiles. These findings highlight how the rate-distortion tradeoff is not a comprehensive lens to understand the behavior of VAEs, and we propose a more mechanistic approach to investigate how the choice of a distortion metric reshapes the optimization problem.

Related papers

• Next-Acceleration-Scale Prediction for Autoregressive MRI Reconstruction [52.32112533846212]
  MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions.<n>We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction.<n>We show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling.
  arXiv  Detail & Related papers  (2026-05-19T04:40:50Z)
• VIKING: Deep variational inference with stochastic projections [48.946143517489496]
  Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks.<n>We propose a simple variational family that considers two independent linear subspaces of the parameter space.<n>This allows us to build a fully-correlated approximate posterior reflecting the overparametrization.
  arXiv  Detail & Related papers  (2025-10-27T15:38:35Z)
• On Self-Adaptive Perception Loss Function for Sequential Lossy Compression [29.361832071511795]
  We consider causal, low-latency, sequential lossy compression, with mean squared-error (MSE) as the distortion loss, and a perception loss function (PLF) to enhance the realism of reconstructions.<n>We establish the theoretical rate-distortion-perception function for first-order Markov sources and analyze the Gaussian model in detail.<n>The proposed metric is referred to as self-adaptive perception loss function (PLF-SA), as its behavior adapts to the quality of reconstructed frames.
  arXiv  Detail & Related papers  (2025-02-15T01:41:53Z)
• PseudoNeg-MAE: Self-Supervised Point Cloud Learning using Conditional Pseudo-Negative Embeddings [55.55445978692678]
  PseudoNeg-MAE enhances global feature representation of point cloud masked autoencoders by making them both discriminative and sensitive to transformations.<n>We propose a novel loss that explicitly penalizes invariant collapse, enabling the network to capture richer transformation cues while preserving discriminative representations.
  arXiv  Detail & Related papers  (2024-09-24T07:57:21Z)
• Unveiling the Ambiguity in Neural Inverse Rendering: A Parameter Compensation Analysis [36.353019226575576]
  Inverse rendering aims to reconstruct the scene properties of objects solely from multiview images. In this paper, we utilize Neural Microfacet Fields (NMF), a state-of-the-art neural inverse rendering method to illustrate the inherent ambiguity.
  arXiv  Detail & Related papers  (2024-04-19T11:56:29Z)
• On the Dynamics Under the Unhinged Loss and Beyond [104.49565602940699]
  We introduce the unhinged loss, a concise loss function, that offers more mathematical opportunities to analyze closed-form dynamics. The unhinged loss allows for considering more practical techniques, such as time-vary learning rates and feature normalization.
  arXiv  Detail & Related papers  (2023-12-13T02:11:07Z)
• DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency [24.5360032541275]
  Diffusion models have established new state of the art in a multitude of computer vision tasks, including image restoration. We propose a novel framework for inverse problem solving, namely we assume that the observation comes from a degradation process that gradually degrades and noises the original clean image. Our technique maintains consistency with the original measurement throughout the reverse process, and allows for great flexibility in trading off perceptual quality for improved distortion metrics and sampling speedup via early-stopping.
  arXiv  Detail & Related papers  (2023-03-25T04:37:20Z)
• Learning Distortion Invariant Representation for Image Restoration from A Causality Perspective [42.10777921339209]
  We propose a novel training strategy for image restoration from the causality perspective. Our method, termed Distortion Invariant representation Learning (DIL), treats each distortion type and degree as one specific confounder.
  arXiv  Detail & Related papers  (2023-03-13T05:04:18Z)
• Occlusion Fields: An Implicit Representation for Non-Line-of-Sight Surface Reconstruction [3.0553868534759725]
  Non-line-of-sight reconstruction (NLoS) aims to recover objects outside the field of view from measurements of light that is indirectly scattered off a directly visible, diffuse wall. We propose a new representation and reconstruction technique for NLoS scenes that unifies the treatment of recoverability with the reconstruction itself.
  arXiv  Detail & Related papers  (2022-03-16T14:47:45Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Limited-angle tomographic reconstruction of dense layered objects by dynamical machine learning [68.9515120904028]
  Limited-angle tomography of strongly scattering quasi-transparent objects is a challenging, highly ill-posed problem. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. We devised a recurrent neural network (RNN) architecture with a novel split-convolutional gated recurrent unit (SC-GRU) as the building block.
  arXiv  Detail & Related papers  (2020-07-21T11:48:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.