GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding

• URL: http://arxiv.org/abs/2605.13352v1
• Date: Wed, 13 May 2026 11:12:18 GMT
• Title: GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding
• Abstract summary: We propose textbfGeoFlowVLM as a post-hoc adapter that learns the joint distribution of paired $ell$-normalised dual-encoder VLM embeddings.<n>A consistency result shows that, in the population limit, the trained network exposes the joint flow and both cross-modal conditional flows.<n>We derive two quantities from this single model: a conditional retrieval entropy that quantifies aleatoric ambiguity with a decision-theoretic interpretation via a Fano-type bound, and a marginaltypicality score justified by an exact chain-rule decomposition of the joint NLL
• Score: 3.0708725114491293
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Standard dual-encoder vision-language models that map images and text to deterministic points on a shared unit hypersphere through $\ell_2$ normalization typically expose neither \emph{aleatoric} uncertainty (cross-modal ambiguity) nor \emph{epistemic} uncertainty (lack of training-distribution support). Existing post-hoc methods either recover at most one of the two uncertainty components, or ignore the hyperspherical geometry of these models' embeddings. We propose \textbf{GeoFlowVLM} as a post-hoc adapter that learns the joint distribution of paired $\ell_2$-normalised dual-encoder VLM embeddings on the product hypersphere $\mathbb{S}^{d-1} \times \mathbb{S}^{d-1}$ via Riemannian flow matching with a single masked velocity field. A consistency result shows that, in the population limit, the trained network exposes the joint flow and both cross-modal conditional flows as valid Riemannian flow-matching velocity fields on their respective domains. We derive two quantities from this single model: a conditional retrieval entropy that quantifies aleatoric ambiguity with a decision-theoretic interpretation via a Fano-type bound, and a marginal-typicality epistemic score justified by an exact chain-rule decomposition of the joint NLL. This decomposition isolates a cross-modal pointwise-mutual-information term that is structurally discriminative rather than epistemic, and is empirically the only consistently uninformative standalone component. Empirically, the entropy tracks Recall@1 with near-ideal monotonic calibration across three retrieval benchmarks in both directions, and the marginal-typicality sum yields consistently calibrated selective accuracy across four zero-shot classification benchmarks.

Related papers

• Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport [88.55403215843967]
  Voronoi-based partial optimal transport (VPOT) for obtaining refined local and global convergence guarantees for the maximum likelihood estimator of the mixing measure.<n>Results hold in arbitrary fixed dimension without requiring mixing proportions to be uniformly bounded away from zero or prior knowledge of the true number of mixture components.
  arXiv  Detail & Related papers  (2026-09-15T04:32:53Z)
• Impute-EM: Native Mixed-State Diffusion Models for Heterogeneous Data Imputation [71.74203645336385]
  Impute-EM alternates between imputing missing entries with the current model and refitting a diffusion backbone on completed data.<n>We characterize the update and show that the observed mask-indexed marginals match the targets at the limit.
  arXiv  Detail & Related papers  (2026-09-14T09:39:13Z)
• Constrained Diffusion Models with Primal-Dual Inference [76.64466170898369]
  We develop constrained diffusion models with primal-dual inference (PDI)<n>PDI infers the optimal primal distribution and its parametrizing dual variable.<n>We evaluate PDI on constrained sampling from a mixture of Gaussians, wireless resource allocation, and portfolio management.
  arXiv  Detail & Related papers  (2026-06-15T18:32:33Z)
• Sobolev Regularized MMD Gradient Flow [22.929984194375805]
  We propose a regularized variant of the gradient-regularized MMD flow.<n>The proposed regularization mitigates the difference between the mean kernel mean embeddings.<n>A key highlight of the proposed flow is that it is applicable in both sampling (from an unnormalized target distribution)
  arXiv  Detail & Related papers  (2026-05-12T09:58:19Z)
• Discrete Flow Matching: Convergence Guarantees Under Minimal Assumptions [13.856153548136826]
  Flow Matching has recently emerged as a popular class of generative models for simulating a target distribution.<n>We study two DFM models on $mathbbZ_md = 0,ldots,m-1d$, sampled through time discretization.<n>We derive non-asymptotic bounds in Kullback--Leibler divergence for the early-stopped version of the target distribution.
  arXiv  Detail & Related papers  (2026-05-09T11:06:33Z)
• Entropy-Controlled Flow Matching [0.08460698440162889]
  We propose a constrained variational principle over continuity-equation paths enforcing a global entropy-rate budget d/dt H(mu_t) >= -lambda.<n>We obtain certificate-style mode-coverage and density-floor guarantees with Lipschitz, and construct near-optimal counterexamples for unconstrained flow matching.
  arXiv  Detail & Related papers  (2026-02-25T06:07:01Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Mean-Field Path-Integral Diffusion: From Samples to Interacting Agents [0.0]
  Mean-Field Path-Integral Diffusion (MF-PID) is a framework in which samples are promoted to interacting agents whose drift depends self-consistently on the evolving population density.<n>MF-PID achieves 19--24% reductions in cumulative control energy over independent-agent baselines while matching the prescribed terminal distribution exactly.
  arXiv  Detail & Related papers  (2026-02-23T21:14:06Z)
• Universality of high-dimensional scaling limits of stochastic gradient descent [8.760293543857706]
  We consider tasks in high dimensions whose loss depends on the data only through its projection into a fixed-dimensional subspace spanned by the parameter vectors and certain ground truth vectors.<n>This includes classifying mixture distributions with cross-entropy loss with one and two-layer networks, and learning single and multi-index models with one and two-layer networks.
  arXiv  Detail & Related papers  (2025-12-15T18:30:26Z)
• Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers [49.97755400231656]
  We present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers.<n>We show that score mismatches result in an distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions.<n>This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise.
  arXiv  Detail & Related papers  (2024-10-17T16:42:12Z)
• Exact Recovery in the General Hypergraph Stochastic Block Model [92.28929858529679]
  This paper investigates fundamental limits of exact recovery in the general d-uniform hypergraph block model (d-HSBM) We show that there exists a sharp threshold such that exact recovery is achievable above the threshold and impossible below it.
  arXiv  Detail & Related papers  (2021-05-11T03:39:08Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.