$β$-VAEs as Effective Theories: Tolerance-Dependent Dimension

• URL: http://arxiv.org/abs/2608.10599v1
• Date: Tue, 11 Aug 2026 07:34:18 GMT
• Title: $β$-VAEs as Effective Theories: Tolerance-Dependent Dimension
• Abstract summary: In a $$-VAE, increasing the regularization strength acts as a spectral cutoff by collapsing low-utility latent coordinates.<n>We find that nonlinear interactions shift and broaden collapse onsets, so thresholds no longer coincide exactly with utilities.<n>The resulting effective-dimension curves reveal a head--tail tradeoff: increasing depth concentrates utility into the first few coordinates but worsens tail fidelity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In a $β$-VAE, increasing the regularization strength acts as a spectral cutoff by collapsing low-utility latent coordinates. In the linear Gaussian VAE, the collapse order matches the ranking of reconstruction utilities exactly, because both are set by the PCA spectrum. We ask which parts of this picture survive in fully connected nonlinear VAEs trained on WorldClim. We find that nonlinear interactions shift and broaden collapse onsets, so thresholds no longer coincide exactly with utilities. However, the common ordering is preserved over the resolved ranks, so the spectral cutoff still acts as a utility cutoff and the effective-description logic carries through. The resulting effective-dimension curves reveal a head--tail tradeoff: increasing depth concentrates utility into the first few coordinates but worsens tail fidelity.

Related papers

• Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL [7.803284875836737]
  bootstrapping can amplify estimation errors in offline reinforcement learning.<n>We propose Convex Hull Neighborhood Smooth Dual Generalization, which expresses the Bellman backup as an in-sample value target plus a CHN-local correction.<n> Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation.
  arXiv  Detail & Related papers  (2026-08-04T04:30:02Z)
• Optimization Geometrodynamics: Variational Reduction and Interaction Curvature [0.0]
  We develop geometrodynamics as a variational theory of this hidden determinant pushforward.<n>For affine pre-reductions, the induced interaction curvature is the negative-semidefinite operator $-G*H-1G$.<n>We prove a global analytic bundle with closed totally geodesic fibers and a unique analytic nearest-controller section.
  arXiv  Detail & Related papers  (2026-07-07T18:42:00Z)
• Posterior Collapse as Automatic Spectral Pruning [0.0]
  We show that posterior collapse in $$-VAEs implements automatic spectral pruning.<n>A latent mode collapses if its contribution to reconstruction is below the cutoff set by $$.<n>We derive this as a consequence of the loss via a Landau stability analysis.
  arXiv  Detail & Related papers  (2026-05-21T16:36:10Z)
• A Theory of Saddle Escape in Deep Nonlinear Networks [0.0]
  We derive an exact identity for the imbalance of Frobenius norms of layer weight matrices that holds for smooth activation and any differentiable loss.<n>On the permutation-symmetric submanifold, the identity combines with an approximate balance law to reduce the full matrix flow to a scalar ODE.<n>We find close agreement between our theory and numerical simulations.
  arXiv  Detail & Related papers  (2026-05-02T06:55:15Z)
• Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction [42.67092904252001]
  We propose a new family of convex loss functions that resolve this tension by stitching a linear core to a smooth tail.<n>We prove that these surrogates are differentiable everywhere while retaining strict linear $H$-consistency bounds.
  arXiv  Detail & Related papers  (2026-04-30T11:32:25Z)
• Regularized Online RLHF with Generalized Bilinear Preferences [68.44113000390544]
  We consider the problem of contextual online RLHF with general preferences.<n>We adopt the Generalized Bilinear Preference Model to capture preferences via low-rank, skew-symmetric matrices.<n>We prove that the dual gap of the greedy policy is bounded by the square of the estimation error.
  arXiv  Detail & Related papers  (2026-02-26T15:27:53Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• MirrorLA: Reflecting Feature Map for Vision Linear Attention [49.41670925034762]
  Linear attention significantly reduces the computational complexity of Transformers from quadratic to linear, yet it consistently lags behind softmax-based attention in performance.<n>We propose MirrorLA, a geometric framework that substitutes passive truncation with active reorientation.<n>MirrorLA achieves state-of-the-art performance across standard benchmarks, demonstrating that strictly linear efficiency can be achieved without compromising representational fidelity.
  arXiv  Detail & Related papers  (2026-02-04T09:14:09Z)
• Error Feedback for Muon and Friends [80.90330715662961]
  We introduce EF21-Muon, the first communication-efficient, non-Euclidean LMO-based with rigorous convergence guarantees.<n>Our theory covers non-Euclidean smooth and the more general $(L0, L1)$-smooth setting, matching best-known Euclidean rates and enabling faster convergence under suitable norm choices.
  arXiv  Detail & Related papers  (2025-10-01T08:20:08Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward [66.81579829897392]
  We propose a novel offline reinforcement learning algorithm called Pessimistic vAlue iteRaTion with rEward Decomposition (PARTED) PARTED decomposes the trajectory return into per-step proxy rewards via least-squares-based reward redistribution, and then performs pessimistic value based on the learned proxy reward. To the best of our knowledge, PARTED is the first offline RL algorithm that is provably efficient in general MDP with trajectory-wise reward.
  arXiv  Detail & Related papers  (2022-06-13T19:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.