A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits

• URL: http://arxiv.org/abs/2605.19944v1
• Date: Tue, 19 May 2026 15:00:51 GMT
• Title: A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits
• Abstract summary: We project discrete trajectories into a metric space to quantify domain shifts.<n>Invoking Kantorovich duality, we bound OOD generalization via architectural Lipschitz continuity and functional approximation limits.
• Score: 17.425526755350948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While empirical scaling laws for LLM reasoning are well-documented, the theoretical mechanisms governing out-of-distribution (OOD) generalization remain elusive. We formalize reasoning via optimal transport, projecting discrete trajectories into a continuous metric space to quantify domain shifts using the Wasserstein-1 distance. Invoking Kantorovich duality, we bound OOD generalization via architectural Lipschitz continuity and functional approximation limits. This exposes two primary constraints. First, position-dependent attention (e.g., Absolute Positional Encoding) fails to preserve shift invariance, yielding an $Ω(1)$ Lipschitz constant and expected risk, whereas shift-invariant mechanisms (e.g., Rotary Embeddings) preserve equivariance and bound the error. Second, by mapping sequential backtracking to a Dyck-$k$ language, we establish a strict circuit depth lower bound for $\text{TC}^0$ Transformers. Scaling physical layer depth is necessary to avert representation collapse -- a constraint that scaling representation width cannot bypass due to irreducible approximation bounds in Barron spaces. Evaluations across 54 Transformer configurations on combinatorial search corroborate these bounds, demonstrating that generalization risk degrades monotonically with the Wasserstein domain shift.

Related papers

• Discretization and Statistical Consistency of Functional Flow Matching [0.0]
  We prove strong $L2$ convergence of finite conditional velocity targets for every strongly consistent sequence of finite-rank reconstructions.<n>For learned flows, coupling directly to a population superposition path yields an end-to-end Wasserstein bound.
  arXiv  Detail & Related papers  (2026-08-05T07:05:47Z)
• A Compositional Theory of Causally Masked Transformers [50.88598486616582]
  We develop a formalization that derives expressivity directly from the model's implemented dynamics.<n>Each attention head updates its own state independently within a layer.<n>A modified form of soft attention supports irreversible, checklist-like state.
  arXiv  Detail & Related papers  (2026-07-29T14:47:19Z)
• Phasor Attention: Mean Root Square Normalization for Phase Manifold Preservation [3.0459410574367314]
  We introduce Mean Root Square Normalization (MRSNorm)<n>MRSNorm mathematically inverts the traditional scaling paradigm.<n>We analytically demonstrate that this constraint yields a built-in, trigonometric gradient clipper governed by the Pythagorean identity.
  arXiv  Detail & Related papers  (2026-07-20T11:10:51Z)
• Nonconvex Composite Functional Constraints via First-Order Augmented Lagrangian Methods under Local Regularity [10.643433352418915]
  We show uniformly augmented condition bounds of primal non-balancing non-concave minimax reformulation.<n>With dual regularization, a global local argument together with a bias.<n>$KKT.
  arXiv  Detail & Related papers  (2026-07-09T21:28:29Z)
• Deforming the Trail: Baseline Quantum Circuitry for $\text{SU(2)}_k$ Lattice Gauge Theory [0.0]
  We provide a strategy of gauge-variant completions to extend unitarity to the entire computational Hilbert space.<n>We show that the physical Hilbert space dimension of the deformed plaquette operator scales equivalently to its non-deformed counterpart with a constant factor.<n>Q-deformation continues to pass scrutiny as a reliable truncation offering advantages in quantum circuit synthesis.
  arXiv  Detail & Related papers  (2026-05-14T17:01:35Z)
• Circuit Complexity of Hierarchical Knowledge Tracing and Implications for Log-Precision Transformers [2.688207424884465]
  Empirically, transformer encoders trained on recursion-invariant trees converge to permutation-invariant shortcuts.<n>These findings motivate structure-aware objectives and iterative mechanisms for prerequisite-sensitive knowledge tracing on deep hierarchies.
  arXiv  Detail & Related papers  (2026-03-25T01:10:42Z)
• Grokking as a Variance-Limited Phase Transition: Spectral Gating and the Epsilon-Stability Threshold [0.0]
  Generalization requires the textitanisotropic rectification unique to adaptives, which directs noise into the tangent space of the solution.<n>This work analyzes AdamW dynamics on modular arithmetic tasks, revealing a Spectral Gating'' mechanism that regulates the transition from generalization to memorization.
  arXiv  Detail & Related papers  (2026-03-16T16:18:18Z)
• Regularized Online RLHF with Generalized Bilinear Preferences [68.44113000390544]
  We consider the problem of contextual online RLHF with general preferences.<n>We adopt the Generalized Bilinear Preference Model to capture preferences via low-rank, skew-symmetric matrices.<n>We prove that the dual gap of the greedy policy is bounded by the square of the estimation error.
  arXiv  Detail & Related papers  (2026-02-26T15:27:53Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Riemannian Flow Matching for Disentangled Graph Domain Adaptation [51.98961391065951]
  Graph Domain Adaptation (GDA) typically uses adversarial learning to align graph embeddings in Euclidean space.<n>DisRFM is a geometry-aware GDA framework that unifies embedding and flow-based transport.
  arXiv  Detail & Related papers  (2026-01-31T11:05:35Z)
• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• Variational Perturbation Theory in Open Quantum Systems for Efficient Steady State Computation [1.5689418447376589]
  Perturbation theory can expand steady states around reference parameters.<n>It relies on the pseudo-inverse and has a limited radius of convergence.<n>We develop two numerical strategies that eliminate the need to compute pseudo-inverses.
  arXiv  Detail & Related papers  (2025-03-31T18:00:01Z)
• Simulating scalar field theories on quantum computers with limited resources [62.997667081978825]
  We present a quantum algorithm for implementing $phi4$ lattice scalar field theory on qubit computers. The algorithm allows efficient $phi4$ state preparation for a large range of input parameters in both the normal and broken symmetry phases.
  arXiv  Detail & Related papers  (2022-10-14T17:28:15Z)
• Barycentric-alignment and reconstruction loss minimization for domain generalization [30.459247038765568]
  This paper advances the theory and practice of Domain Generalization (DG) in machine learning. We propose a novel DG algorithm named Wasserstein Barycenter Auto-Encoder (WBAE) that simultaneously minimizes the classification loss, the barycenter loss, and the reconstruction loss. Numerical results demonstrate that the proposed method outperforms current state-of-the-art DG algorithms on several datasets.
  arXiv  Detail & Related papers  (2021-09-04T16:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.