Phase transitions for the noisy transformer model in arbitrary dimension

• URL: http://arxiv.org/abs/2606.05140v1
• Date: Wed, 03 Jun 2026 17:49:43 GMT
• Title: Phase transitions for the noisy transformer model in arbitrary dimension
• Abstract summary: We study the McKean--Vlasov free energy on the unit sphere associated with the unnormalized self-attention (USA) model for noisy transformer dynamics.<n>We prove a sharp global-minimizer dichotomy in every dimension $dge2$.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the McKean--Vlasov free energy on the unit sphere associated with the unnormalized self-attention (USA) model for noisy transformer dynamics. We prove a sharp global-minimizer dichotomy in every dimension $d\ge2$. There is a unique $β_*^{(d)}>0$ such that \begin{equation*} \frac{I_{d/2+1}(β_*^{(d)})}{I_{d/2}(β_*^{(d)})}=\frac1d, \end{equation*} where $I_ν$ is the modified Bessel function of the first kind. For $0<β\le β_*^{(d)}$, the uniform density remains the unique global minimizer up to the linear-stability threshold \begin{equation*} K_\#^{(d)}(β)=\frac{β^{d/2}}{2^{d/2}Γ(d/2)I_{d/2}(β)}, \end{equation*} and the phase transition is continuous. For $β>β_*^{(d)}$, the uniform density is not globally minimizing at $K_\#^{(d)}(β)$, so the critical coupling satisfies $K_c<K_\#^{(d)}(β)$ and the transition is discontinuous. This result generalizes the authors' recent $d=2$ work arXiv:2604.16288 to arbitrary dimension. The proof uses the sharp Beckner--Onofri/logarithmic Hardy-Littlewood-Sobolev (HLS) inequality on the sphere, together with a Funk--Hecke/Bessel coefficient computation and a degree-two quartic obstruction.

Related papers

• Quantitative Diffusive Limits for Singular Nonlocal Transport [11.229917469094966]
  We study the nonlocal continuity equation [ partial_t_b =operatornamediv!!left( _bnablalogbigl((I-b2)-1_bbigr).<n>For smooth strictly positive initial data, we prove that as $b to 0$, its global solution converges to heat flow $(t)$ at the sharp, uniform-in-time rate.
  arXiv  Detail & Related papers  (2026-09-10T17:29:01Z)
• Wave-particle duality as an uncertainty relation for the average confidence width [0.0]
  We show the confidence width $_c x(_x)$ -- the smallest position interval carrying a fraction $_x$ of the probability -- averaged over all levels.<n>A mean-entropy argument with the Bialynicki-Birula-Mycielski relation gives the rigorous $cge/e$, while the achievable constant $cast$ is set by the ground state of the Fourier-invariant operator $|x|+|p|$.
  arXiv  Detail & Related papers  (2026-06-30T10:19:35Z)
• Tikhonov-regularised projected gradient flow for equality-constrained bilinear quantum control [0.0]
  We study a projection-type gradient flow for equality-constrained control objective on $mathcalH=L(0,T;mathbbR)$.<n>We replace $$ with $_varepsilon:=+varepsilon2I$ and prove: (i) an exact spectral identity giving $(_varepsilon)=(_min2+varepsilon2)$; (ii) objective monotonicity $mathrmdJ/
  arXiv  Detail & Related papers  (2026-04-29T12:53:58Z)
• Phase transitions in Doi-Onsager, Noisy Transformer, and other multimodal models [0.0]
  We study phase transitions for repulsive-attractive mean-field free energies on the circle.<n>We prove that the critical coupling strength $K_c$ coincides with the linear stability threshold $K_#$ of the uniform distribution.
  arXiv  Detail & Related papers  (2026-04-17T17:50:29Z)
• Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition [50.36362492608702]
  We study entrywise scalar quantization of two matrices prior to multiplication.<n>We obtain a closed-form optimal point density [ star(u) propto exp!left(-fracu26right)bigl( (1-2)+2u22bigr), qquad u=fracx_X,, and prove a correlation-driven phase transition.
  arXiv  Detail & Related papers  (2026-03-20T01:53:44Z)
• Rényi exponent landscape of multipartite entanglement in free-fermion systems [51.56484100374058]
  We show that the Rényi tripartite information $I_3() exhibits a qualitatively $exclusion-dependent scaling at small Fermi momentum.<n>A direct consequence is a replica obstruction: $I_m(n)/I_m(1) sim zm-1 to 0$ for all integer $n geq 2$, so the leading von Neumann signal cannot be reconstructed from integer Rényi data.
  arXiv  Detail & Related papers  (2026-03-09T22:27:00Z)
• Quantum Brownian motion with non-Gaussian noises: Fluctuation-Dissipation Relation and nonlinear Langevin equation [0.0]
  We consider the quantum Brownian motion (QBM) model with one oscillator as the system.<n>The influence action $S_IF$ is calculated using a perturbative expansion in $$.<n>The non-Gaussian noise kernel gives rise to non-zero three-point correlation function of the corresponding force.
  arXiv  Detail & Related papers  (2026-02-11T02:03:26Z)
• On the $O(\frac{\sqrt{d}}{T^{1/4}})$ Convergence Rate of RMSProp and Its Momentum Extension Measured by $\ell_1$ Norm [54.28350823319057]
  This paper considers the RMSProp and its momentum extension and establishes the convergence rate of $frac1Tsum_k=1T.<n>Our convergence rate matches the lower bound with respect to all the coefficients except the dimension $d$.<n>Our convergence rate can be considered to be analogous to the $frac1Tsum_k=1T.
  arXiv  Detail & Related papers  (2024-02-01T07:21:32Z)
• A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing [68.80803866919123]
  Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $mathbfx*$ rather than for all $mathbfx*$ simultaneously. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy.
  arXiv  Detail & Related papers  (2023-09-25T17:54:19Z)
• Measurement-induced phase transition for free fermions above one dimension [46.176861415532095]
  Theory of the measurement-induced entanglement phase transition for free-fermion models in $d>1$ dimensions is developed. Critical point separates a gapless phase with $elld-1 ln ell$ scaling of the second cumulant of the particle number and of the entanglement entropy.
  arXiv  Detail & Related papers  (2023-09-21T18:11:04Z)
• Anharmonic oscillator: a solution [77.34726150561087]
  The dynamics in $x$-space and in $(gx)-space corresponds to the same energy spectrum with effective coupling constant $hbar g2$. A 2-classical generalization leads to a uniform approximation of the wavefunction in $x$-space with unprecedented accuracy.
  arXiv  Detail & Related papers  (2020-11-29T22:13:08Z)
• Optimal Mean Estimation without a Variance [103.26777953032537]
  We study the problem of heavy-tailed mean estimation in settings where the variance of the data-generating distribution does not exist. We design an estimator which attains the smallest possible confidence interval as a function of $n,d,delta$.
  arXiv  Detail & Related papers  (2020-11-24T22:39:21Z)
• R\'{e}nyi and Tsallis entropies of the Dirichlet and Neumann one-dimensional quantum wells [0.0]
  Dirichlet and Neumann boundary conditions (BCs) of 1D quantum well are studied. For either BC the dependencies of the R'enyi position components on the parameter $alpha$ are the same for all orbitals. The gap between the thresholds $alpha_TH$ of the two BCs causes different behavior of the R'enyi uncertainty relations as functions of $alpha$.
  arXiv  Detail & Related papers  (2020-03-09T18:34:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.