The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Models

• URL: http://arxiv.org/abs/2604.04155v1
• Date: Sun, 05 Apr 2026 15:45:49 GMT
• Title: The Geometric Alignment Tax: Tokenization vs. Continuous Geometry in Scientific Foundation Models
• Abstract summary: We identify the root cause: the Geometric Alignment Tax.<n>We evaluate 14 biological foundation models with rate-distortion theory and MINE.<n>No model achieves simultaneously low distortion, high mutual information, and global coherence.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models for biology and physics optimize predictive accuracy, but their internal representations systematically fail to preserve the continuous geometry of the systems they model. We identify the root cause: the Geometric Alignment Tax, an intrinsic cost of forcing continuous manifolds through discrete categorical bottlenecks. Controlled ablations on synthetic dynamical systems demonstrate that replacing cross-entropy with a continuous head on an identical encoder reduces geometric distortion by up to 8.5x, while learned codebooks exhibit a non-monotonic double bind where finer quantization worsens geometry despite improving reconstruction. Under continuous objectives, three architectures differ by 1.3x; under discrete tokenization, they diverge by 3,000x. Evaluating 14 biological foundation models with rate-distortion theory and MINE, we identify three failure regimes: Local-Global Decoupling, Representational Compression, and Geometric Vacuity. A controlled experiment confirms that Evo 2's reverse-complement robustness on real DNA reflects conserved sequence composition, not learned symmetry. No model achieves simultaneously low distortion, high mutual information, and global coherence.

Related papers

• Reliability-Aware 3D Geometric Injection for Universal Person Re-identification [47.874819213762926]
  We propose a Universal Monocular 3D-Enhanced ReID framework driven by a Consistency-Aware Reliability Gate and Dual-Stream Residual Fusion.<n>Our method improves challenging, structure-sensitive scenarios while preserving competitive performance on clean domains.
  arXiv  Detail & Related papers  (2026-07-21T08:54:25Z)
• Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI [0.7766379063545538]
  We introduce the Statistically Meaningful Geometry (SMG) framework, an information-geometric paradigm lifting parametric models into infinite-dimensional minimization non-asymptotic statistics.<n>We prove that under connection-filtered pre-training, out-of-distribution predictive variance is strictly upper-bounded by the finite diameter of the identifiable base manifold $mathcalB$, establishing a hard geometric containment of generative hallucinations.
  arXiv  Detail & Related papers  (2026-07-03T13:46:31Z)
• Statistically Meaningful Geometry and Gauge Symmetry Breaking: A Geometric Foundation for Scientific Discovery and Intelligence Emergence [0.7766379063545538]
  Do machine learning systems exhibit genuine intelligence, or are they merely sophisticated statistical pattern matchers?<n>We introduce Statistically Meaningful Geometry, a framework modeling over- parameterized learning systems as infinite-dimensional non-parametric Orlicz fiber bundles.<n>We show that SMG provides a parameter-free, falsifiable dashboard to mathematically certify true intelligence, transforming AI for Science into an engine of autonomous paradigm shifts.
  arXiv  Detail & Related papers  (2026-07-03T13:34:56Z)
• Engineering Quantum Many-Body Scars through Lattice Geometry [0.0028001975341488856]
  We show that lattice geometry alone can serve as a powerful and experimentally accessible control for inducing and enhancing scarring.<n>By transforming a one-dimensional chain into a quasi-one-dimensional triangle-decorated lattice, we find that the fully polarized state exhibits pronounced fidelity revivals, slow entanglement growth, and strong overlap with a tower of weakly entangled eigenstates.
  arXiv  Detail & Related papers  (2026-05-06T18:00:01Z)
• SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments [75.60795462502949]
  SpatialEvo is a self-evolving framework for 3D spatial reasoning.<n>It formalizes 16 spatial reasoning task categories under explicit geometric validation rules.<n>It converts unannotated 3D scenes into zero-noise interactive oracles, replacing model consensus with objective physical feedback.
  arXiv  Detail & Related papers  (2026-04-15T17:59:12Z)
• The Geometric Price of Discrete Logic: Context-driven Manifold Dynamics of Number Representations [6.414826816896125]
  Large language models (LLMs) generalize smoothly across continuous semantic spaces, yet strict logical reasoning demands the formation of discrete decision boundaries.<n>We argue that task context operates as a non-isometric dynamical operator that enforces a necessary "topological distortion"<n>We validate this geometric evolution across a gradient of tasks, from simple mapping to complex primality testing.
  arXiv  Detail & Related papers  (2026-03-24T13:41:57Z)
• Measuring 3D Spatial Geometric Consistency in Dynamic Generated Videos [67.7364297817535]
  We introduce SGC, a metric for evaluating 3D textbfSpatial textbfGeometric textbfConsistency in videos.<n>SGC robustly quantifies geometric inconsistencies, effectively identifying critical failures missed by existing metrics.
  arXiv  Detail & Related papers  (2026-03-19T15:44:39Z)
• Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation [56.361076943802594]
  CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.
  arXiv  Detail & Related papers  (2026-02-16T18:58:55Z)
• The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-Modal Divergence [17.501700376593174]
  We present a measure-theoretic framework that models learning as the evolution of representation measures on a fixed embedding manifold.<n>By establishing value and consistency in the large-batch limit, we bridge the misalignment objective to explicit energy landscapes.<n>We show that this term induces barrier-driven co-adaptation, enforcing a population-level modality gap as a structural geometric necessity.
  arXiv  Detail & Related papers  (2026-01-27T13:33:03Z)
• Multi-Scale Negative Coupled Information Systems (MNCIS): A Unified Spectral Topology Framework for Stability in Turbulence, AI, and Biology [1.4213973379473657]
  This work generalizes the Multi-Scale Negative Coupled Information System (MNCIS) framework.<n>Global stability requires an active topological operator -- Adaptive Spectral Negative Coupling (ASNC) -- functioning as a state-dependent high-pass filter.<n>ASNC acts as a global-enstrophy adaptive sub-grid scale (SGS) model, stabilizing the inviscid limit and preserving the Kolmogorov $-5/3$ inertial range without artificial hyper-viscosity.<n>Our results suggest that the MNCIS framework provides a base-independent topological condition for distinguishing viable complex systems from those collapsing into thermal equilibrium
  arXiv  Detail & Related papers  (2026-01-06T21:11:33Z)
• Dynamics of Agentic Loops in Large Language Models: A Geometric Theory of Trajectories [0.0]
  This paper introduces a geometric framework for analyzing agentic trajectories in semantic embedding space.<n>Because cosine similarity is biased by embedding anisotropy, we introduce an isotonic calibration.<n>This enables rigorous measurement of trajectories, clusters and attractors.
  arXiv  Detail & Related papers  (2025-12-11T07:06:14Z)
• Geometric Meta-Learning via Coupled Ricci Flow: Unifying Knowledge Representation and Quantum Entanglement [7.410691988131121]
  This paper establishes a unified framework integrating geometric flows with deep learning through three fundamental innovations.<n>First, we propose a thermodynamically coupled Ricci flow that dynamically adapts parameter space geometry to loss landscape topology.<n>Second, we derive explicit phase transition thresholds and critical learning rates through curvature blowup analysis.<n>Third, we establish an AdS/CFT-type holographic duality (Theoremrefthm:ads) between neural networks and conformal field theories.
  arXiv  Detail & Related papers  (2025-03-25T17:32:31Z)
• Geometric Trajectory Diffusion Models [58.853975433383326]
  Generative models have shown great promise in generating 3D geometric systems. Existing approaches only operate on static structures, neglecting the fact that physical systems are always dynamic in nature. We propose geometric trajectory diffusion models (GeoTDM), the first diffusion model for modeling the temporal distribution of 3D geometric trajectories.
  arXiv  Detail & Related papers  (2024-10-16T20:36:41Z)
• Learning Physical Dynamics with Subequivariant Graph Neural Networks [99.41677381754678]
  Graph Neural Networks (GNNs) have become a prevailing tool for learning physical dynamics. Physical laws abide by symmetry, which is a vital inductive bias accounting for model generalization. Our model achieves on average over 3% enhancement in contact prediction accuracy across 8 scenarios on Physion and 2X lower rollout MSE on RigidFall.
  arXiv  Detail & Related papers  (2022-10-13T10:00:30Z)
• Geometric phase in a dissipative Jaynes-Cummings model: theoretical explanation for resonance robustness [68.8204255655161]
  We compute the geometric phases acquired in both unitary and dissipative Jaynes-Cummings models. In the dissipative model, the non-unitary effects arise from the outflow of photons through the cavity walls. We show the geometric phase is robust, exhibiting a vanishing correction under a non-unitary evolution.
  arXiv  Detail & Related papers  (2021-10-27T15:27:54Z)
• Quantum anomalous Hall phase in synthetic bilayers via twistless twistronics [58.720142291102135]
  We propose quantum simulators of "twistronic-like" physics based on ultracold atoms and syntheticdimensions. We show that our system exhibits topologicalband structures under appropriate conditions.
  arXiv  Detail & Related papers  (2020-08-06T19:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.