Functional Bias and Tangent-Space Geometry in Variational Inference

• URL: http://arxiv.org/abs/2603.08925v1
• Date: Mon, 09 Mar 2026 20:52:14 GMT
• Title: Functional Bias and Tangent-Space Geometry in Variational Inference
• Abstract summary: We show that the leading-order bias of a posterior functional is determined by its component to the variational tangent space induced by the variational family.<n>We further derive explicit expansions for the bias of functionals and show that omitted interaction directions produce first-order distortion of cross-block dependence measures.
• Score: 0.12183405753834557
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Variational inference approximates Bayesian posterior distributions by projecting onto a tractable family of distributions. While most theoretical analyses evaluate the quality of this approximation using global divergence measures, many applications rely on specific posterior summaries such as expectations, variances, or tail probabilities. We develop a geometric framework for analyzing the bias of posterior functionals under variational approximations. We show that the leading-order bias of a posterior functional is determined by its component orthogonal to the variational tangent space induced by the variational family. Functionals aligned with this space incur only second-order bias. For structured mean-field variational families we characterize the tangent space explicitly and show that it consists of block-additive functions of the parameter blocks, while interaction components determine the leading-order bias. Under standard local asymptotic normality conditions we further derive explicit asymptotic expansions for the bias of posterior functionals and show that omitted interaction directions produce first-order distortion of cross-block dependence measures. These results provide a geometric explanation for several well-known properties of mean-field variational inference, including the systematic distortion of cross-block dependencies.

Related papers

• Intrinsic Riemannian Cross-covariance for Manifold-valued Random Objects [6.722667861696959]
  This paper introduces an intrinsic Riemannian cross-covariance for manifold-valued random objects.<n>Our approach defines covariance and correlation by transporting local variations to a common tangent space via parallel transport.<n>We establish that the proposed covariance inherits desirable properties of its Euclidean counterparts and characterize its behavior.
  arXiv  Detail & Related papers  (2026-06-08T22:05:16Z)
• On Higher-Order Geometric Refinements of Classical Covariance Asymptotics: An Approach via Intrinsic and Extrinsic Information Geometry [0.0]
  In curved models, including mixtures, curved exponential families, latent-variable models, and manifold-root parameter spaces, finite-sample behavior can deviate systematically from predictions.<n>We develop a coordinate-invariant, curvature-aware refinement by viewing a regular parametric family as a ((,g)) with Fisher-constrained-Rao metric.<n>We describe the resolved metric, the role of the real log canonical threshold in learning rates and posterior mean-squared error, and a curvature-based covariance expansion on the resolved space that recovers the regular theory as a special case.
  arXiv  Detail & Related papers  (2026-04-14T13:40:13Z)
• Implicit bias as a Gauge correction: Theory and Inverse Design [2.9379512315137117]
  A central problem in machine learning theory is to characterize how learning dynamics select particular solutions compatible with the training objective.<n>We identify a general mechanism, in terms of an explicit correction of the learning dynamics, for the emergence of implicit biases.<n>We compute the resulting induced bias for a range of dynamics, showing how several well known results fit into a single unified framework.
  arXiv  Detail & Related papers  (2026-01-10T15:33:09Z)
• Theory and computation for structured variational inference [13.739142487196034]
  We prove the first results for existence, uniqueness, and self-consistency of the variational approximation.<n>We develop a gradient-based algorithm with provable guarantees for computing the variational approximation.
  arXiv  Detail & Related papers  (2025-11-13T02:57:54Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Geometric Prior Guided Feature Representation Learning for Long-Tailed Classification [47.09355487357069]
  We propose to leverage the geometric information of the feature distribution of the well-represented head class to guide the model to learn the underlying distribution of the tail class. It aims to make the perturbed features cover the underlying distribution of the tail class as much as possible, thus improving the model's generalization performance in the test domain.
  arXiv  Detail & Related papers  (2024-01-21T09:16:29Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Spectrum-Aware Debiasing: A Modern Inference Framework with Applications to Principal Components Regression [1.342834401139078]
  We introduce SpectrumAware Debiasing, a novel method for high-dimensional regression. Our approach applies to problems with structured, heavy tails, and low-rank structures. We demonstrate our method through simulated and real data experiments.
  arXiv  Detail & Related papers  (2023-09-14T15:58:30Z)
• On the renormalization group fixed point of the two-dimensional Ising model at criticality [77.34726150561087]
  We show that a simple, explicit analytic description of a fixed point using operator-algebraic renormalization (OAR) is possible. Specifically, the fixed point is characterized in terms of spin-spin correlation functions.
  arXiv  Detail & Related papers  (2023-04-06T16:57:28Z)
• Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency [53.90687548731265]
  We study optimal procedures for estimating a linear functional based on observational data. For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
  arXiv  Detail & Related papers  (2023-01-16T02:57:37Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Amortized backward variational inference in nonlinear state-space models [0.0]
  We consider the problem of state estimation in general state-space models using variational inference. We establish for the first time that, under mixing assumptions, the variational approximation of expectations of additive state functionals induces an error which grows at most linearly in the number of observations.
  arXiv  Detail & Related papers  (2022-06-01T08:35:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.