Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction

• URL: http://arxiv.org/abs/2605.20107v1
• Date: Tue, 19 May 2026 16:57:47 GMT
• Title: Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction
• Abstract summary: JEPAs often regularize one-view embeddings toward an isotropic Gaussian, implicitly baking Euclidean symmetry into the representation.<n>We show that this is not merely a benign default. When the downstream geometry is unknown, no geometry-independent fixed marginal target is canonical.<n>We instantiate this principle with textbfHamJEPA, which encodes each view as a phase-space state $(q,p)$ and predicts view-to-view transitions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: JEPAs often regularize one-view embeddings toward an isotropic Gaussian, implicitly baking Euclidean symmetry into the representation. We show that this is not merely a benign default. For a known structured downstream geometry $H\succ0$, the minimax and maximum-entropy covariance under a Hamiltonian energy budget is $(c/d)H^{-1}$, and Euclidean isotropy incurs a closed-form price of isotropy. More importantly, when the downstream geometry is unknown, no geometry-independent fixed marginal target is canonical: every fixed covariance shape can be maximally misaligned for some structured geometry. We further show that even oracle one-view marginals do not identify the JEPA view-to-view predictive coupling. These results suggest that the structural bias in JEPAs should enter the cross-view coupling rather than a fixed encoder marginal. We instantiate this principle with \textbf{HamJEPA}, which encodes each view as a phase-space state $(q,p)$ and predicts view-to-view transitions with a learned Hamiltonian leapfrog map, while non-isotropic scale and spectral floors prevent collapse. In a deliberately headless token protocol, HamJEPA improves over SIGReg on CIFAR-100 by $+4.89$ kNN@20 and $+3.52$ linear-probe points at 30 epochs, and by $+6.45$ kNN@20 and $+10.64$ linear-probe points at 80 epochs, while a matched MLP predictor ablation shows that the symplectic coupling is the ingredient driving the neighborhood-geometry gain. On ImageNet-100, HamJEPA-$q$ improves by $+4.82$ kNN@20 and $+7.52$ linear-probe points at 45 epochs.

Related papers

• $\mathbb{SL}(n)$ Representation Learning: An Intrinsic Mixed-Curvature Space with Higher Curvature Capacities and Deeper Order-Aware Composition [47.80325814873361]
  Mixed-curvature representation learning seeks to capture rich geometric structures that cannot be adequately modeled by a single curvature regime.<n>We introduce the $mathbbSL(n)$ space, a representation geometry defined by the simple $det(A)variant=1$ constraint and a left in Schatten-$p$ structure.<n>Despite this minimal construction, $mathbbSL(n)$ exhibits pointwise negative, zero, and flag curvature around a common flagpole.
  arXiv  Detail & Related papers  (2026-09-14T05:52:14Z)
• Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality [0.0]
  Kolmogorov-Arnold Networks (KANs) replace fixed activations in deep architectures.<n>Family of edge activations derived from Banach duality maps.<n>Learnable exponents provide an interpretable, relative signal.
  arXiv  Detail & Related papers  (2026-08-26T13:53:13Z)
• Demixing Sparse Signals from Nonlinear Observations using Generalized Non-convex Regularization [0.0]
  We consider the recovery of a pair of sparse vectors from a limited number of nonlinear observations of their superposition.<n>We propose a regularization-based framework combining a Huberized data fidelity with generalized folded-concave penalties.<n>We derive estimation error bounds of order $sqrtslog(n)/m$ that hold at emphevery localized stationary point.
  arXiv  Detail & Related papers  (2026-07-12T07:29:39Z)
• Terminal Dimension Reduction for Time Series with Applications [53.49083917849734]
  Terminal embeddings have emerged as a powerful tool for dimension reduction.<n>We develop a generalization of terminal embeddings to affine line-segments that overcomes this issue.<n>We obtain the first dimension-free coresets for clustering time-series under the Fréchet distance.
  arXiv  Detail & Related papers  (2026-07-10T15:05:30Z)
• UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures [0.0]
  We propose emphUR-JEPA, which targets a uniformly $n$-rectifiable measure of local dimension tangent $n$ at small scales.<n>We show that on all four datasets UR--JEPA($mathcalLtextCGLT$) produces a global geometric spectrum with a $4$ to $5$ order-of-magnitude drop at index $sim 20$ to $25$ out of $D = 32$, while LeJEPA's spectrum is near-flat (top-to-bottom ratio at most $3.6$)
  arXiv  Detail & Related papers  (2026-05-31T20:26:54Z)
• Hidden-State Privacy Has an Empty Middle [51.56484100374058]
  We prove a complementary Fisher-ball lower bound: every full-rank Gaussian release at $O(1)$ Fisher utility admits a direction whose Mahalanobis signal grows linearly in hidden width.<n>A split-memory transformer trained from scratch reaches $G_mathrmMah in [20, 33]$ at 90M and maintains a $6$--$24times$ advantage over same-budget GPT baselines from 30M to 1B at a fixed-token language loss penalty.
  arXiv  Detail & Related papers  (2026-05-21T20:12:09Z)
• Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds [61.14405512940818]
  We show that Score-based generative models are trained in high-dimensional ambient spaces.<n>We use a ReLU implementation of nearest-projection coordinates via finite intrinsic anchors and Gauss Newton.
  arXiv  Detail & Related papers  (2026-05-15T10:20:05Z)
• OAM-Induced Lattice Rotation Reveals a Fractional Optimum in Fault-Tolerant GKP Quantum Sensing [0.0]
  We establish that orbital-angular-momentum encoding and Gottesman-Kitaev-Preskill (GKP) lattice geometry are structurally coupled.<n>Using an end-to-end differentiable Strawberry Fields--TensorFlow circuit, we jointly optimise $ell$, the lattice aspect ratio $r$, and the finite-energy envelope $$ to maximise quantum Fisher information.
  arXiv  Detail & Related papers  (2026-05-13T09:49:16Z)
• Efficient Logistic Regression with Mixture of Sigmoids [15.14559143371469]
  We show that the near-optimal worst-case regret bound $O(dlog(Bn))$ for EW can be achieved with total worst-case computational complexity $O(B3 n5)$.<n>Our results show that EW can be both computationally tractable and geometrically adaptive in online classification.
  arXiv  Detail & Related papers  (2026-04-03T09:36:34Z)
• Information Routing in Atomistic Foundation Models: How Task Alignment and Equivariance Shape Linear Disentanglement [0.0]
  We introduce Compositional Probe Decomposition (CPD), which linearly projects out composition signal and measures how much geometric information remains accessible to a Ridge probe.<n>We find a emphlinear accessibility gradient: models differ by $6.6times$ in geometric information accessible after composition removal.<n>We also show that nonlinear probes produce misleading results on residualized representations, recovering $R2 = 0.68$--$0.95$ on a purely compositional target, and recommend linear probes for this setting.
  arXiv  Detail & Related papers  (2026-03-03T16:52:06Z)
• A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing [68.80803866919123]
  Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $mathbfx*$ rather than for all $mathbfx*$ simultaneously. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy.
  arXiv  Detail & Related papers  (2023-09-25T17:54:19Z)
• Measurement-induced phase transition for free fermions above one dimension [46.176861415532095]
  Theory of the measurement-induced entanglement phase transition for free-fermion models in $d>1$ dimensions is developed. Critical point separates a gapless phase with $elld-1 ln ell$ scaling of the second cumulant of the particle number and of the entanglement entropy.
  arXiv  Detail & Related papers  (2023-09-21T18:11:04Z)
• Near-optimal fitting of ellipsoids to random points [68.12685213894112]
  A basic problem of fitting an ellipsoid to random points has connections to low-rank matrix decompositions, independent component analysis, and principal component analysis. We resolve this conjecture up to logarithmic factors by constructing a fitting ellipsoid for some $n = Omega(, d2/mathrmpolylog(d),)$. Our proof demonstrates feasibility of the least squares construction of Saunderson et al. using a convenient decomposition of a certain non-standard random matrix.
  arXiv  Detail & Related papers  (2022-08-19T18:00:34Z)
• Kondo effect in a non-Hermitian, $\mathcal{PT}$-symmetric Anderson model with Rashba spin-orbit coupling [3.1013276969076666]
  We show that for a non-Hermitian hybridization, $lambda$ can renormalize the exceptional point even in the non-interacting case. We show that in the strong coupling regime, $lambda$ and interactions co-operate in strongly reducing the critical point associated with Kondo destruction.
  arXiv  Detail & Related papers  (2022-01-01T12:03:47Z)
• Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness [151.67113334248464]
  We show that extending the smoothing technique to defend against other attack models can be challenging. We present experimental results on CIFAR to validate our theory.
  arXiv  Detail & Related papers  (2020-02-08T22:02:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.