Posterior Collapse as a Phase Transition in Variational Autoencoders

• URL: http://arxiv.org/abs/2510.01621v1
• Date: Thu, 02 Oct 2025 02:52:25 GMT
• Title: Posterior Collapse as a Phase Transition in Variational Autoencoders
• Authors: Zhen Li, Fan Zhang, Zheng Zhang, Yu Chen,
• Abstract summary: We investigate the phenomenon of posterior collapse in variational autoencoders (VAEs) from the perspective of statistical physics.<n>By analyzing the stability of the trivial solution associated with posterior collapse, we identify a critical hyper- parameter threshold.<n>We validate this critical behavior on both synthetic and real-world datasets, confirming the existence of a phase transition.
• Score: 13.161084138023169
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate the phenomenon of posterior collapse in variational autoencoders (VAEs) from the perspective of statistical physics, and reveal that it constitutes a phase transition governed jointly by data structure and model hyper-parameters. By analyzing the stability of the trivial solution associated with posterior collapse, we identify a critical hyper-parameter threshold. This critical boundary, separating meaningful latent inference from collapse, is characterized by a discontinuity in the KL divergence between the approximate posterior and the prior distribution. We validate this critical behavior on both synthetic and real-world datasets, confirming the existence of a phase transition. Our results demonstrate that posterior collapse is not merely an optimization failure, but rather an emerging phase transition arising from the interplay between data structure and variational constraints. This perspective offers new insights into the trainability and representational capacity of deep generative models.

Related papers

• Manifold Percolation: from generative model to Reinforce learning [0.26905021039717986]
  Generative modeling is typically framed as learning mapping rules, but from an observer's perspective without access to these rules, the task becomes disentangling the geometric support from the probability distribution.<n>We propose that continuum percolation is uniquely suited to this support analysis, as the sampling process effectively projects high-dimensional density estimation onto a geometric counting problem on the support.
  arXiv  Detail & Related papers  (2025-11-25T17:12:42Z)
• VIKING: Deep variational inference with stochastic projections [48.946143517489496]
  Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks.<n>We propose a simple variational family that considers two independent linear subspaces of the parameter space.<n>This allows us to build a fully-correlated approximate posterior reflecting the overparametrization.
  arXiv  Detail & Related papers  (2025-10-27T15:38:35Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Topological Phase Transitions and Mixed State Order in a Hubbard Quantum Simulator [36.556659404501914]
  Topological phase transitions challenge conventional paradigms in many-body physics.<n>We observe such a transition between one-dimensional crystalline symmetry-protected topological phases.<n>Our results demonstrate how topology and information influence quantum phase transitions.
  arXiv  Detail & Related papers  (2025-05-22T17:58:35Z)
• A Study of Posterior Stability for Time-Series Latent Diffusion [59.41969496514184]
  We first show that posterior collapse will reduce latent diffusion to a variational autoencoder (VAE), making it less expressive. We then introduce a principled method: dependency measure, that quantifies the sensitivity of a recurrent decoder to input variables. Building on our theoretical and empirical studies, we introduce a new framework that extends latent diffusion and has a stable posterior.
  arXiv  Detail & Related papers  (2024-05-22T21:54:12Z)
• Score-based Causal Representation Learning with Interventions [54.735484409244386]
  This paper studies the causal representation learning problem when latent causal variables are observed indirectly. The objectives are: (i) recovering the unknown linear transformation (up to scaling) and (ii) determining the directed acyclic graph (DAG) underlying the latent variables.
  arXiv  Detail & Related papers  (2023-01-19T18:39:48Z)
• Steady-state susceptibility in continuous phase transitions of dissipative systems [3.0429703764855343]
  We find that the susceptibilities of fidelity and trace distance exhabit singular behaviors near the critical points of phase transitions in both models. The critical points, in thermodynamic limit, extracted from the scalings of the critical controlling parameters to the system size or nonlinearity agree well with the existed results.
  arXiv  Detail & Related papers  (2022-01-12T11:56:52Z)
• Variational Causal Networks: Approximate Bayesian Inference over Causal Structures [132.74509389517203]
  We introduce a parametric variational family modelled by an autoregressive distribution over the space of discrete DAGs. In experiments, we demonstrate that the proposed variational posterior is able to provide a good approximation of the true posterior.
  arXiv  Detail & Related papers  (2021-06-14T17:52:49Z)
• Stochastic embeddings of dynamical phenomena through variational autoencoders [1.7205106391379026]
  We use a recognition network to increase the observed space dimensionality during the reconstruction of the phase space. Our validation shows that this approach not only recovers a state space that resembles the original one, but it is also able to synthetize new time series.
  arXiv  Detail & Related papers  (2020-10-13T10:10:24Z)
• Fidelity susceptibility near topological phase transitions in quantum walks [0.0]
  We show that for topological phase transitions in Dirac models, the fidelity susceptibility coincides with the curvature function whose integration gives the topological invariant. We map out the profile and criticality of the fidelity susceptibility to quantum walks that simulate one-dimensional class BDI and two-dimensional class D Dirac models.
  arXiv  Detail & Related papers  (2020-07-21T09:11:52Z)
• Supporting Optimal Phase Space Reconstructions Using Neural Network Architecture for Time Series Modeling [68.8204255655161]
  We propose an artificial neural network with a mechanism to implicitly learn the phase spaces properties. Our approach is either as competitive as or better than most state-of-the-art strategies.
  arXiv  Detail & Related papers  (2020-06-19T21:04:47Z)
• Discrete Variational Attention Models for Language Generation [51.88612022940496]
  We propose a discrete variational attention model with categorical distribution over the attention mechanism owing to the discrete nature in languages. Thanks to the property of discreteness, the training of our proposed approach does not suffer from posterior collapse.
  arXiv  Detail & Related papers  (2020-04-21T05:49:04Z)
• Topological Phase Transitions Induced by Varying Topology and Boundaries in the Toric Code [0.0]
  We study the sensitivity of such phases of matter to the underlying topology. We claim that these phase transitions are accompanied by broken symmetries in the excitation space. We show that the phase transition between such steady states is effectively captured by the expectation value of the open-loop operator.
  arXiv  Detail & Related papers  (2020-04-07T18:00:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.