The Hyperspherical Geometry of CLIP Latent Space: A Semantic Mixture Model

• URL: http://arxiv.org/abs/2607.13660v1
• Date: Wed, 15 Jul 2026 10:05:53 GMT
• Title: The Hyperspherical Geometry of CLIP Latent Space: A Semantic Mixture Model
• Abstract summary: Contrastive Language-Image Pretraining (CLIP) representations form a semantic embedding space governed by cosine similarity.<n>We propose a principled density model for the CLIP latent space based on Mixtures of von Mises-Fisher (MovMF) distributions.
• Score: 73.29820477295043
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive Language-Image Pretraining (CLIP) representations form a semantic embedding space governed by cosine similarity, reflecting an intrinsic hyperspherical geometry. However, existing probabilistic interpretations typically rely on Gaussian assumptions, which fail to capture this directional and multimodal structure. We propose a principled density model for the CLIP latent space based on Mixtures of von Mises-Fisher (MovMF) distributions defined on the unit hypersphere. Using the Expectation-Maximization (EM) algorithm, we efficiently learn a probabilistic model in which each mixture component corresponds to a coherent semantic concept. This formulation yields a closed-form likelihood naturally aligned with hyperspherical geometry, enabling accurate and interpretable density estimation. Empirically, our model significantly improves long-tailed and out-of-distribution detection and provides a natural semantic decomposition, representing each embedding as a sparse probabilistic combination of interpretable concepts. These results suggest that CLIP latent space is more faithfully characterized as a hyperspherical semantic mixture rather than an isotropic Gaussian, establishing a simple and geometrically consistent probabilistic framework for modeling and understanding multimodal representations. Project page is available at https://xiaoyuzhizi.github.io/movmf-clip/.

Related papers

• Geometry-Aware Dataset Condensation for Diffusion Model Training [103.45641113998839]
  We propose to reformulate real subset selection as a geometry-aware distribution alignment problem.<n>By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution.<n>Experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage.
  arXiv  Detail & Related papers  (2026-06-04T08:53:58Z)
• Representational Capacity: Geometric Limits on Feature Representation in Transformer Language Models [51.56484100374058]
  We develop a framework for estimating how many near-orthogonal directions a model can support.<n>Applying this metric across dozens of open-source models reveals two classes: models with high $varepsilon$ whose embeddings lack near-orthogonal structure, and models with low $varepsilon$ that maintain it.
  arXiv  Detail & Related papers  (2026-06-01T18:28:56Z)
• The Normalized Maximum Likelihood for Regular Non-Smooth Models: Measure-Theoretic Foundations and Geometric Sampling [0.0]
  We provide a rigorous framework for computing the Normalized Maximum Likelihood (NML) codelength for non-smooth models.<n>By applying classical geometric measure theory and bridging the coarea formula with conservative Jacobians, we prove that the complexity for non-smooth models is well-posed.<n>Our work paves the way for the theoretical analysis of the NML codelength for regular non-smooth models.
  arXiv  Detail & Related papers  (2026-05-23T08:57:48Z)
• A Geometric Unification of Generative AI with Manifold-Probabilistic Projection Models [3.870354915766566]
  In some generative models, the low dimensional nature of the data manifest itself by the introduction of a lower dimensional latent space.<n>This study unifies the geometric and probabilistic perspectives by providing a geometric framework and a kernel-based probabilistic method simultaneously.<n>We demonstrate that the Latent MPPM (LMPPM) outperforms the Latent Diffusion Model (LDM) across various datasets.
  arXiv  Detail & Related papers  (2025-10-01T08:50:30Z)
• Learning Overspecified Gaussian Mixtures Exponentially Fast with the EM Algorithm [5.625796693054093]
  We investigate the convergence properties of the EM algorithm when applied to overspecified Gaussian mixture models.<n>We demonstrate that the population EM algorithm converges exponentially fast in terms of the Kullback-Leibler (KL) distance.
  arXiv  Detail & Related papers  (2025-06-13T14:57:57Z)
• QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Prediction [49.75084732129701]
  3D occupancy prediction is crucial for robust autonomous driving systems.<n>Most existing methods employ dense voxel-based scene representations.<n>We present QuadricFormer, a superquadric-based model for efficient 3D occupancy prediction.
  arXiv  Detail & Related papers  (2025-06-12T17:59:45Z)
• Learning over von Mises-Fisher Distributions via a Wasserstein-like Geometry [0.0]
  We introduce a geometry-aware distance metric for the family of von Mises-Fisher (vMF) distributions.<n>Motivated by the theory of optimal transport, we propose a Wasserstein-like distance that decomposes the discrepancy between two vMF distributions into two interpretable components.
  arXiv  Detail & Related papers  (2025-04-19T03:38:15Z)
• Injective flows for star-like manifolds [1.4623202528810306]
  We show that we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs, for star-like manifold densities.<n>This is particularly relevant for variational inference settings, where no samples are available and only some unnormalized target is known.
  arXiv  Detail & Related papers  (2024-06-13T13:43:59Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Kernel Density Matrices for Probabilistic Deep Learning [8.486487001779416]
  In quantum mechanics, a density matrix is the most general way to describe the state of a quantum system. This paper introduces a novel approach to probabilistic deep learning, kernel density matrices. It provides a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables.
  arXiv  Detail & Related papers  (2023-05-26T12:59:58Z)
• Optimal Scaling for Locally Balanced Proposals in Discrete Spaces [65.14092237705476]
  We show that efficiency of Metropolis-Hastings (M-H) algorithms in discrete spaces can be characterized by an acceptance rate that is independent of the target distribution. Knowledge of the optimal acceptance rate allows one to automatically tune the neighborhood size of a proposal distribution in a discrete space, directly analogous to step-size control in continuous spaces.
  arXiv  Detail & Related papers  (2022-09-16T22:09:53Z)
• Moser Flow: Divergence-based Generative Modeling on Manifolds [49.04974733536027]
  Moser Flow (MF) is a new class of generative models within the family of continuous normalizing flows (CNF) MF does not require invoking or backpropagating through an ODE solver during training. We demonstrate for the first time the use of flow models for sampling from general curved surfaces.
  arXiv  Detail & Related papers  (2021-08-18T09:00:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.