High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models

• URL: http://arxiv.org/abs/2204.04476v1
• Date: Sat, 9 Apr 2022 14:03:21 GMT
• Title: High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models
• Abstract summary: We provide a "path-wise" characterization of the overlap between the output of the Langevin algorithm and the planted signal. As a second contribution, we derive an explicit formula for the limiting overlap in terms of the signal-to-noise ratio and the injected noise in the diffusion.
• Score: 3.712310463095685
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study Langevin dynamics for recovering the planted signal in the spiked matrix model. We provide a "path-wise" characterization of the overlap between the output of the Langevin algorithm and the planted signal. This overlap is characterized in terms of a self-consistent system of integro-differential equations, usually referred to as the Crisanti-Horner-Sommers-Cugliandolo-Kurchan (CHSCK) equations in the spin glass literature. As a second contribution, we derive an explicit formula for the limiting overlap in terms of the signal-to-noise ratio and the injected noise in the diffusion. This uncovers a sharp phase transition -- in one regime, the limiting overlap is strictly positive, while in the other, the injected noise overcomes the signal, and the limiting overlap is zero.

Related papers

• Stochastic Scaling Limits and Synchronization by Noise in Deep Transformer Models [6.60601600774899]
  We prove pathwise convergence of the layerwise evolution of tokens in a finite-depth, finite-width transformer with Multi-Layer Perceptron (MLP) blocks.<n>We also identify the partial differential equation describing the evolution of the tokens' distribution in this limit and prove propagation of chaos when the number of such tokens is large.
  arXiv  Detail & Related papers  (2026-04-29T17:09:05Z)
• Random Matrix Theory of Early-Stopped Gradient Flow: A Transient BBP Scenario [1.2744523252873352]
  We show that learning occurs when an isolated eigenvalue separates from a noisy bulk, before disappearing in the overfitting regime.<n>We map the corresponding phase diagrams and validate the theory against finite-size simulations.
  arXiv  Detail & Related papers  (2026-04-20T16:05:58Z)
• Stochastic Gradient Descent in the Saddle-to-Saddle Regime of Deep Linear Networks [74.46751089984072]
  gradient descent (SGD) noise on descent in deep linear networks is poorly understood.<n>We investigate the dynamics of SGD during training of DLNs in the saddle-to-saddle regime.<n>Our results establish that SGD noise encodes information about the progression of feature learning but does not fundamentally alter the saddle-to-saddle dynamics.
  arXiv  Detail & Related papers  (2026-04-07T18:43:08Z)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking [49.1352577985191]
  We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task.<n>Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics.
  arXiv  Detail & Related papers  (2026-02-18T20:25:13Z)
• Contraction and entropy production in continuous-time Sinkhorn dynamics [0.6423239719448169]
  We give an exact identity for the entropy production rate of the Sinkhorn flow, which was previously known only to be nonpositive.<n>We show that the flow induces a reversible Markov dynamics on the target marginal as an Onsager gradient flow.<n>We give for illustration two immediate practical use-cases for the Sinkhorn LSI: as a design principle for the latent space in which generative models are trained, and as a stopping algorithm for discrete-time algorithms.
  arXiv  Detail & Related papers  (2025-10-14T15:32:15Z)
• Contractive kinetic Langevin samplers beyond global Lipschitz continuity [0.0]
  We show that two novel discretizations of the kinetic Langevin SDE satisfy a log-Sobolev inequality.<n>We establish a series of non-asymptotic bounds in $2$-Wasserstein distance between the law reached by each algorithm and the underlying measure.
  arXiv  Detail & Related papers  (2025-09-15T15:14:45Z)
• Spin-only dynamics of the multi-species nonreciprocal Dicke model [0.0]
  Hepp-Lieb-Dicke model is ubiquitous in cavity quantum electrodynamics.<n>We study a variation of the open Dicke model which realizes mediated nonreciprocal interactions between spin species.<n>We find signatures of phase transitions even for small system sizes.
  arXiv  Detail & Related papers  (2025-07-10T17:41:46Z)
• Information scrambling and entanglement dynamics in Floquet Time Crystals [49.1574468325115]
  We study the dynamics of out-of-time-ordered correlators (OTOCs) and entanglement of entropy as measures of information propagation in disordered systems.
  arXiv  Detail & Related papers  (2024-11-20T17:18:42Z)
• Exact dynamics of quantum dissipative $XX$ models: Wannier-Stark localization in the fragmented operator space [49.1574468325115]
  We find an exceptional point at a critical dissipation strength that separates oscillating and non-oscillating decay. We also describe a different type of dissipation that leads to a single decay mode in the whole operator subspace.
  arXiv  Detail & Related papers  (2024-05-27T16:11:39Z)
• Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems [78.96969465641024]
  We extend mean-field Langevin dynamics to minimax optimization over probability distributions for the first time with symmetric and provably convergent updates. We also study time and particle discretization regimes and prove a new uniform-in-time propagation of chaos result.
  arXiv  Detail & Related papers  (2023-12-02T13:01:29Z)
• Dynamical transitions from slow to fast relaxation in random open quantum systems [0.0]
  We study a model in which the system Hamiltonian and its couplings to the noise are random matrices whose entries decay as power laws of distance. The steady state is always featureless, but the rate at which it is approached exhibits three phases depending on $alpha_H$ and $alpha_L$. Within perturbation theory, the phase boundaries in the $(alpha_H, alpha_L)$ plane differ for weak and strong dissipation, suggesting phase transitions as a function of noise strength.
  arXiv  Detail & Related papers  (2022-11-23T20:56:46Z)
• Hybrid discrete-continuous truncated Wigner approximation for driven, dissipative spin systems [0.0]
  We present a systematic approach for the treatment of many-body dynamics of interacting, open spin systems. Our approach overcomes some of the shortcomings of the recently developed discrete truncated Wigner approximation (DTWA) We show that the continuous embedding allows for a straightforward extension of the method to open spin systems subject to dephasing, losses and incoherent drive.
  arXiv  Detail & Related papers  (2022-03-31T15:40:18Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Rotating Majorana Zero Modes in a disk geometry [75.34254292381189]
  We study the manipulation of Majorana zero modes in a thin disk made from a $p$-wave superconductor. We analyze the second-order topological corner modes that arise when an in-plane magnetic field is applied. We show that oscillations persist even in the adiabatic phase because of a frequency independent coupling between zero modes and excited states.
  arXiv  Detail & Related papers  (2021-09-08T11:18:50Z)
• Spread of Correlations in Strongly Disordered Lattice Systems with Long-Range Coupling [0.0]
  We investigate the spread of correlations carried by an excitation in a 1-dimensional lattice system with high on-site energy disorder. The increase in correlation between the initially quenched node and a given node exhibits three phases: quadratic in time, linear in time, and saturation.
  arXiv  Detail & Related papers  (2021-06-15T15:47:20Z)
• Continuous and time-discrete non-Markovian system-reservoir interactions: Dissipative coherent quantum feedback in Liouville space [62.997667081978825]
  We investigate a quantum system simultaneously exposed to two structured reservoirs. We employ a numerically exact quasi-2D tensor network combining both diagonal and off-diagonal system-reservoir interactions with a twofold memory for continuous and discrete retardation effects. As a possible example, we study the non-Markovian interplay between discrete photonic feedback and structured acoustic phononovian modes, resulting in emerging inter-reservoir correlations and long-living population trapping within an initially-excited two-level system.
  arXiv  Detail & Related papers  (2020-11-10T12:38:35Z)
• Tuning the topology of $p$-wave superconductivity in an analytically solvable two-band model [0.0]
  We introduce and solve a two-band model of spinless fermions with $p_x$-wave pairing on a square lattice. We show that its phase diagram contains a topologically nontrivial weak pairing phase as well as a trivial strong pairing phase.
  arXiv  Detail & Related papers  (2020-10-01T01:20:46Z)
• Discrete truncated Wigner approach to dynamical phase transitions in Ising models after a quantum quench [0.0]
  We study dynamical phase transitions arising in the steady state of transverse-field Ising models after a quantum quench. We find identical exponents for $alpha lesssim 0.5$, suggesting that the dynamical transitions in this regime fall into the same universality class as the nonergodic mean-field limit.
  arXiv  Detail & Related papers  (2020-04-21T08:20:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.