Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry

• URL: http://arxiv.org/abs/2511.01064v1
• Date: Sun, 02 Nov 2025 20:10:57 GMT
• Title: Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry
• Abstract summary: We provide symmetry-based guarantees for variational inference with location-scale families.<n>First, we provide guarantees for a broader family of divergences, highlighting the properties of variational objectives under which VI provably recovers the mean and correlation matrix.<n>Second, we obtain further guarantees for VI when the target density $p$ exhibits even and elliptical symmetries in some but not all of its coordinates.
• Score: 3.9655818200495734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We extend several recent results providing symmetry-based guarantees for variational inference (VI) with location-scale families. VI approximates a target density~$p$ by the best match $q^*$ in a family $Q$ of tractable distributions that in general does not contain $p$. It is known that VI can recover key properties of $p$, such as its mean and correlation matrix, when $p$ and $Q$ exhibit certain symmetries and $q^*$ is found by minimizing the reverse Kullback-Leibler divergence. We extend these guarantees in two important directions. First, we provide symmetry-based guarantees for a broader family of divergences, highlighting the properties of variational objectives under which VI provably recovers the mean and correlation matrix. Second, we obtain further guarantees for VI when the target density $p$ exhibits even and elliptical symmetries in some but not all of its coordinates. These partial symmetries arise naturally in Bayesian hierarchical models, where the prior induces a challenging geometry but still possesses axes of symmetry. We illustrate these theoretical results in a number of experimental settings.

Related papers

• Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration [71.51826063983434]
  Image restoration is an inherently ill posed inverse problem.<n>We use non strict symmetry as a constraint to formulate the restoration inverse problem.<n>We show that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size.
  arXiv  Detail & Related papers  (2026-05-13T16:22:19Z)
• Tight Convergence Rates for Online Distributed Linear Estimation with Adversarial Measurements [66.94250413799232]
  We study mean estimation of a random vector $X$ in a distributed parameter-server-worker setup.<n>Key challenges are adversarial measurements and asynchrony.<n>Results provide a unified finite-time characterization of robustness, identifiability, and statistical efficiency in distributed linear estimation with adversarial workers.
  arXiv  Detail & Related papers  (2026-04-07T11:45:55Z)
• Regularized Online RLHF with Generalized Bilinear Preferences [68.44113000390544]
  We consider the problem of contextual online RLHF with general preferences.<n>We adopt the Generalized Bilinear Preference Model to capture preferences via low-rank, skew-symmetric matrices.<n>We prove that the dual gap of the greedy policy is bounded by the square of the estimation error.
  arXiv  Detail & Related papers  (2026-02-26T15:27:53Z)
• Breaking global symmetries with locality-preserving operations [0.0]
  We show that locality-preserving operations can generate maximal asymmetry.<n>Our results highlight a non-trivial interplay between asymmetry, locality, and entanglement.
  arXiv  Detail & Related papers  (2025-08-21T18:00:02Z)
• Mesoscopic Fluctuations and Multifractality at and across Measurement-Induced Phase Transition [46.176861415532095]
  We explore statistical fluctuations over the ensemble of quantum trajectories in a model of two-dimensional free fermions.<n>Our results exhibit a remarkable analogy to Anderson localization, with $G_AB$ corresponding to two-terminal conductance.<n>Our findings lay the groundwork for mesoscopic theory of monitored systems, paving the way for various extensions.
  arXiv  Detail & Related papers  (2025-07-15T13:44:14Z)
• Variational Inference in Location-Scale Families: Exact Recovery of the Mean and Correlation Matrix [3.9655818200495734]
  Given an intractable target density $p$, variational inference (VI) attempts to find the best approximation $q$ from a tractable family $Q$.<n>In practice, $Q$ is not rich enough to contain $p$, and the approximation is misspecified even when it is a unique global minimizer of $textKL(q||p)$.<n>We prove strong guarantees for VI not only under mild regularity conditions but also in the face of severe misspecifications.
  arXiv  Detail & Related papers  (2024-10-14T20:25:49Z)
• Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks [43.88179780450706]
  We investigate the impact of weight space permutation symmetries on variational inference. We devise a symmetric symmetrization mechanism for constructing permutation invariant variational posteriors. We show that the symmetrized distribution has a strictly better fit to the true posterior, and that it can be trained using the original ELBO objective.
  arXiv  Detail & Related papers  (2024-08-10T09:06:34Z)
• Remarks on effects of projective phase on eigenstate thermalization hypothesis [0.0]
  We consider $mathbbZ_NtimesmathbbZ_N$ symmetries with nontrivial projective phases. We also perform numerical analyses for $ (1+1)$-dimensional spin chains and the $ (2+1)$-dimensional lattice gauge theory.
  arXiv  Detail & Related papers  (2023-10-17T17:36:37Z)
• How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization [46.55524654398093]
  We show how over- parameterization changes the convergence behaviors of descent. The goal is to recover an unknown low-rank ground-rank ground-truth matrix from near-isotropic linear measurements. We propose a novel method that only modifies one step of GD and obtains a convergence rate independent of $alpha$.
  arXiv  Detail & Related papers  (2023-10-03T03:34:22Z)
• Transformers as Support Vector Machines [54.642793677472724]
  We establish a formal equivalence between the optimization geometry of self-attention and a hard-margin SVM problem. We characterize the implicit bias of 1-layer transformers optimized with gradient descent. We believe these findings inspire the interpretation of transformers as a hierarchy of SVMs that separates and selects optimal tokens.
  arXiv  Detail & Related papers  (2023-08-31T17:57:50Z)
• Searching for exceptional points and inspecting non-contractivity of trace distance in (anti-)$\mathcal{PT}\!-$symmetric systems [0.0]
  Non-Hermitian systems with parity-time ($mathcalPT$) symmetry and anti-$mathcalPT$ symmetry give rise to exceptional points (EPs) We propose a powerful and easily computable tool, based on the Hilbert-Schmidt speed (HSS), which does not require the diagonalization of the evolved density matrix. We find that the trace distance, a measure of distinguishability of two arbitrary quantum states, may be non-contractive under the non-Hermitian evolution of the system.
  arXiv  Detail & Related papers  (2021-01-12T18:42:52Z)
• Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry [9.695960412426672]
  We analytically characterize the Hessian at various families of spurious minima. In particular, we prove that for $dge k$ standard Gaussian inputs: (a) of the $dk$ eigenvalues of the Hessian, $dk - O(d)$ concentrate near zero, (b) $Omega(d)$ of the eigenvalues grow linearly with $k$.
  arXiv  Detail & Related papers  (2020-08-04T20:08:35Z)
• A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent [85.77233010209368]
  This article characterizes the exacts of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples $n$ is all large and comparable. This analysis also provides accurate estimates of training and test regression errors for large $n,p,N$.
  arXiv  Detail & Related papers  (2020-06-09T02:05:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.