Propagation of Chaos in Contextual Flow Maps

• URL: http://arxiv.org/abs/2605.16747v1
• Date: Sat, 16 May 2026 02:03:20 GMT
• Title: Propagation of Chaos in Contextual Flow Maps
• Abstract summary: We develop a statistical theory of transformers in the large-context regime by adopting the abstraction of contextual flow maps.<n>We exploit the McKean--Vlasov structure of the dynamics and the classical machinery of propagation of chaos.
• Score: 9.424822243203105
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a quantitative statistical theory of transformers in the large-context regime by adopting the abstraction of contextual flow maps (CFMs): dynamical systems that evolve a distinguished token in the presence of a contextual measure across a stack of attention blocks. Within this framework, the finite-context model approximates an idealized infinite-context system in which the contextual measure is replaced by its underlying population, so that the context length $n$ becomes a statistical resource. Exploiting the McKean--Vlasov structure of the dynamics and the classical machinery of propagation of chaos, we establish a forward bound controlling the deviation between the finite- and infinite-context CFMs uniformly along depth, and a backward bound controlling the deviation between the corresponding training trajectories uniformly across iterations of online gradient descent. Both bounds achieve the optimal Wasserstein rate $n^{-1/d}$ for general CFMs and parametric rate $n^{-1/2}$ for a restricted class of CFMs that includes transformers as a special case. The analysis rests on a new Eulerian adjoint formulation of the loss gradient and stability estimates for the resulting forward--adjoint system, both of which may be of independent interest.

Related papers

• Nonlocality-induced critical-length hierarchy from non-Hermitian competition [48.9823527611878]
  In short-range systems formed by two coupled chains, competition leads to the critical-length law $N_csimln D$.<n>Here we show that long-range hoppings fundamentally reorganizes this critical behavior, producing a hierarchy of distinct scaling laws.<n>Our results offer a platform-independent framework testable in programmable topoelectrical circuits, photonic lattices and digital quantum simulators.
  arXiv  Detail & Related papers  (2026-08-03T18:00:18Z)
• Entropy-Controlled Flow Matching [0.08460698440162889]
  We propose a constrained variational principle over continuity-equation paths enforcing a global entropy-rate budget d/dt H(mu_t) >= -lambda.<n>We obtain certificate-style mode-coverage and density-floor guarantees with Lipschitz, and construct near-optimal counterexamples for unconstrained flow matching.
  arXiv  Detail & Related papers  (2026-02-25T06:07:01Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• A scalable flow-based approach to mitigate topological freezing [34.54607280864912]
  We present a flow-based strategy to remove topological artifacts from Markov Chain Monte Carlo simulations.<n>The strategy is based on a Normalizing Flow (SNF) that alternates non-equilibrium Monte Carlo updates with localized, stout-equivariant defect layers.<n>We show that defect SNFs achieve better performances than reproducing non-equilibrium methods at comparable cost.
  arXiv  Detail & Related papers  (2026-01-28T15:40:46Z)
• Noise-induced decoherence-free zones for anyons [0.0]
  We develop a framework for anyonic systems in which the exchange phase is promoted from a fixed parameter to a fluctuating quantity.<n>We show that the protected mode always minimizes its dephasing at $thetastar = pi/2$, independent of the specific form of $D$.<n>This highlights a simple design rule for optimizing coherence in noisy anyonic systems.
  arXiv  Detail & Related papers  (2025-10-07T16:21:57Z)
• Revisiting boundary-driven method for transport: Finite-size effects and the role of system-bath coupling [0.0]
  We compare and contrasting the dc diffusion constant $mathcalD_textdc$ computed from two approaches.<n>We find a clear mismatch between the two, with $mathcalD_textdc$ exhibiting a strong dependence on the system-bath coupling.<n>As a practical resolution, we advocate computing only time-dependent transport coefficients within the boundary-driven framework.
  arXiv  Detail & Related papers  (2025-07-22T12:35:03Z)
• Scaling of Stochastic Normalizing Flows in $\mathrm{SU}(3)$ lattice gauge theory [44.99833362998488]
  Non-equilibrium Markov Chain Monte Carlo simulations provide a well-understood framework based on Jarzynski's equality to sample from a target probability distribution.<n>Out-of-equilibrium evolutions share the same framework of flow-based approaches and they can be naturally combined into a novel architecture called Normalizing Flows (SNFs)<n>We present the first implementation of SNFs for $mathrmSU(3)$ lattice gauge theory in 4 dimensions, defined by introducing gauge-equivariant layers between out-of-equilibrium Monte Carlo updates.
  arXiv  Detail & Related papers  (2024-11-29T19:01:05Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• Super-model ecosystem: A domain-adaptation perspective [101.76769818069072]
  This paper attempts to establish the theoretical foundation for the emerging super-model paradigm via domain adaptation. Super-model paradigms help reduce computational and data cost and carbon emission, which is critical to AI industry.
  arXiv  Detail & Related papers  (2022-08-30T09:09:43Z)
• Efficient CDF Approximations for Normalizing Flows [64.60846767084877]
  We build upon the diffeomorphic properties of normalizing flows to estimate the cumulative distribution function (CDF) over a closed region. Our experiments on popular flow architectures and UCI datasets show a marked improvement in sample efficiency as compared to traditional estimators.
  arXiv  Detail & Related papers  (2022-02-23T06:11:49Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Optical Flow Estimation [72.41370576242116]
  We propose a semi-supervised Feature Pyramidal Correlation and Residual Reconstruction Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of two main modules: pyramid correlation mapping and residual reconstruction. Experiment results show that the proposed scheme achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and 0.10 in terms of average end-point error (AEE) against competing baseline methods.
  arXiv  Detail & Related papers  (2020-01-17T07:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.