Variational Geometric Information Bottleneck: Learning the Shape of Understanding

• URL: http://arxiv.org/abs/2511.02496v1
• Date: Tue, 04 Nov 2025 11:33:54 GMT
• Title: Variational Geometric Information Bottleneck: Learning the Shape of Understanding
• Authors: Ronald Katende,
• Abstract summary: Variational Geometric Information Bottleneck (V-GIB) is a variational estimator that unifies mutual-information compression and curvature regularization.<n>V-GIB provides a principled and measurable route to representations that are geometrically coherent, data-efficient, and aligned with human-understandable structure.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a unified information-geometric framework that formalizes understanding in learning as a trade-off between informativeness and geometric simplicity. An encoder phi is evaluated by U(phi) = I(phi(X); Y) - beta * C(phi), where C(phi) penalizes curvature and intrinsic dimensionality, enforcing smooth, low-complexity manifolds. Under mild manifold and regularity assumptions, we derive non-asymptotic bounds showing that generalization error scales with intrinsic dimension while curvature controls approximation stability, directly linking geometry to sample efficiency. To operationalize this theory, we introduce the Variational Geometric Information Bottleneck (V-GIB), a variational estimator that unifies mutual-information compression and curvature regularization through tractable geometric proxies such as the Hutchinson trace, Jacobian norms, and local PCA. Experiments across synthetic manifolds, few-shot settings, and real-world datasets (Fashion-MNIST, CIFAR-10) reveal a robust information-geometry Pareto frontier, stable estimators, and substantial gains in interpretive efficiency. Fractional-data experiments on CIFAR-10 confirm that curvature-aware encoders maintain predictive power under data scarcity, validating the predicted efficiency-curvature law. Overall, V-GIB provides a principled and measurable route to representations that are geometrically coherent, data-efficient, and aligned with human-understandable structure.

Related papers

• Understanding and Improving UMAP with Geometric and Topological Priors: The JORC-UMAP Algorithm [1.7484982792736636]
  dimensionality reduction techniques, particularly UMAP, are widely used for visualizing high-dimensional data.<n>We introduce Ollivier-Ricci curvature as a geometric prior, reinforcing edges at geometric bottlenecks and reducing redundant links.<n>Experiments on synthetic and real-world datasets show that JORC-UMAP reduces tearing and collapse more effectively than standard UMAP and other DR methods.
  arXiv  Detail & Related papers  (2026-01-23T08:42:56Z)
• Causal Manifold Fairness: Enforcing Geometric Invariance in Representation Learning [0.0]
  We introduce Causal Manifold Fairness (CMF), a novel framework that bridges causal inference and geometric deep learning.<n>By enforcing constraints on the Jacobian and Hessian of the decoder, CMF ensures that the rules of the latent space are preserved across demographic groups.<n>We validate CMF on synthetic Structural Causal Models (SCMs), demonstrating that it effectively disentangles sensitive geometric warping while preserving task utility.
  arXiv  Detail & Related papers  (2026-01-06T14:05:22Z)
• An approach to Fisher-Rao metric for infinite dimensional non-parametric information geometry [0.6138671548064355]
  Being infinite dimensional, non-parametric information geometry has long faced an "intractability barrier"<n>This paper introduces a novel framework to resolve the intractability with an Orthogonal Decomposition of the Tangent Space.<n>By defining the Information Capture Ratio, we provide a rigorous method for estimating intrinsic dimensionality in high-dimensional data.
  arXiv  Detail & Related papers  (2025-12-25T00:18:41Z)
• CP$^2$: Leveraging Geometry for Conformal Prediction via Canonicalization [51.716834831684004]
  We study the problem of conformal prediction (CP) under geometric data shifts.<n>We propose integrating geometric information--such as geometric pose--into the conformal procedure to reinstate its guarantees.
  arXiv  Detail & Related papers  (2025-06-19T10:12:02Z)
• Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models [63.331590876872944]
  We propose a method for deriving Riemannian metrics directly from pretrained Energy-Based Models.<n>These metrics define spatially varying distances, enabling the computation of geodesics.<n>We show that EBM-derived metrics consistently outperform established baselines.
  arXiv  Detail & Related papers  (2025-05-23T12:18:08Z)
• What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold [0.053713376045563095]
  We introduce a score-based Riemannian metric to characterize the intrinsic geometry of a data manifold.<n>Our approach creates a geometry where geodesics naturally follow the manifold's contours.<n>We show that our score-based geodesics capture meaningful perpendicular transformations that respect the underlying data distribution.
  arXiv  Detail & Related papers  (2025-05-16T11:19:57Z)
• Gradient-Based Feature Learning under Structured Data [57.76552698981579]
  In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction. We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue. In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
  arXiv  Detail & Related papers  (2023-09-07T16:55:50Z)
• SIGMA: Scale-Invariant Global Sparse Shape Matching [50.385414715675076]
  We propose a novel mixed-integer programming (MIP) formulation for generating precise sparse correspondences for non-rigid shapes. We show state-of-the-art results for sparse non-rigid matching on several challenging 3D datasets.
  arXiv  Detail & Related papers  (2023-08-16T14:25:30Z)
• CAMEL: Curvature-Augmented Manifold Embedding and Learning [21.945022912830044]
  Curvature-Augmented Manifold Embedding and Learning (CAMEL) is proposed for high dimensional data classification, dimension reduction, and visualization. CAMEL has been evaluated on various benchmark datasets and has shown to outperform state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-03-05T03:12:50Z)
• Efficient Semi-Implicit Variational Inference [65.07058307271329]
  We propose an efficient and scalable semi-implicit extrapolational (SIVI) Our method maps SIVI's evidence to a rigorous inference of lower gradient values.
  arXiv  Detail & Related papers  (2021-01-15T11:39:09Z)
• Manifold Learning via Manifold Deflation [105.7418091051558]
  dimensionality reduction methods provide a valuable means to visualize and interpret high-dimensional data. Many popular methods can fail dramatically, even on simple two-dimensional Manifolds. This paper presents an embedding method for a novel, incremental tangent space estimator that incorporates global structure as coordinates. Empirically, we show our algorithm recovers novel and interesting embeddings on real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2020-07-07T10:04:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.