Neural Networks as Spin Models: From Glass to Hidden Order Through Training

• URL: http://arxiv.org/abs/2408.06421v1
• Date: Mon, 12 Aug 2024 18:01:04 GMT
• Title: Neural Networks as Spin Models: From Glass to Hidden Order Through Training
• Authors: Richard Barney, Michael Winer, Victor Galitksi,
• Abstract summary: We explore a one-to-one correspondence between a neural network (NN) and a statistical mechanical spin model. We study the magnetic phases and the melting transition temperature as training progresses.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We explore a one-to-one correspondence between a neural network (NN) and a statistical mechanical spin model where neurons are mapped to Ising spins and weights to spin-spin couplings. The process of training an NN produces a family of spin Hamiltonians parameterized by training time. We study the magnetic phases and the melting transition temperature as training progresses. First, we prove analytically that the common initial state before training--an NN with independent random weights--maps to a layered version of the classical Sherrington-Kirkpatrick spin glass exhibiting a replica symmetry breaking. The spin-glass-to-paramagnet transition temperature is calculated. Further, we use the Thouless-Anderson-Palmer (TAP) equations--a theoretical technique to analyze the landscape of energy minima of random systems--to determine the evolution of the magnetic phases on two types of NNs (one with continuous and one with binarized activations) trained on the MNIST dataset. The two NN types give rise to similar results, showing a quick destruction of the spin glass and the appearance of a phase with a hidden order, whose melting transition temperature $T_c$ grows as a power law in training time. We also discuss the properties of the spectrum of the spin system's bond matrix in the context of rich vs. lazy learning. We suggest that this statistical mechanical view of NNs provides a useful unifying perspective on the training process, which can be viewed as selecting and strengthening a symmetry-broken state associated with the training task.

Related papers

• Speak so a physicist can understand you! TetrisCNN for detecting phase transitions and order parameters [3.669506968635671]
  TetrisCNN is a convolutional NN with parallel branches using different kernels that detects the phases of spin systems. We show that TetrisCNN can detect more complex order parameters using the example of two-dimensional Ising gauge theory. This work can lead to the integration of NNs with quantum simulators to study new exotic phases of matter.
  arXiv  Detail & Related papers  (2024-11-04T16:30:58Z)
• Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems [77.88054335119074]
  We use FNOs to model the evolution of random quantum spin systems. We apply FNOs to a compact set of Hamiltonian observables instead of the entire $2n$ quantum wavefunction.
  arXiv  Detail & Related papers  (2024-09-05T07:18:09Z)
• SpinMultiNet: Neural Network Potential Incorporating Spin Degrees of Freedom with Multi-Task Learning [0.0]
  This study introduces SpinMultiNet, a novel NNP model that integrates spin degrees of freedom through multi-task learning. SpinMultiNet achieves accurate predictions without relying on correct spin values obtained from DFT calculations. validation on a dataset of transition metal oxides demonstrates the high predictive accuracy of SpinMultiNet.
  arXiv  Detail & Related papers  (2024-09-05T05:13:28Z)
• Neural network approach to quasiparticle dispersions in doped antiferromagnets [0.0]
  We study the ability of neural quantum states to represent the bosonic and fermionic $t-J$ model on different 1D and 2D lattices. We present a method to calculate dispersion relations from the neural network state representation.
  arXiv  Detail & Related papers  (2023-10-12T17:59:33Z)
• Speed Limits for Deep Learning [67.69149326107103]
  Recent advancement in thermodynamics allows bounding the speed at which one can go from the initial weight distribution to the final distribution of the fully trained network. We provide analytical expressions for these speed limits for linear and linearizable neural networks. Remarkably, given some plausible scaling assumptions on the NTK spectra and spectral decomposition of the labels -- learning is optimal in a scaling sense.
  arXiv  Detail & Related papers  (2023-07-27T06:59:46Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• General time-reversal equivariant neural network potential for magnetic materials [5.334610924852583]
  This study introduces time-reversal E(3)-equivariant neural network and SpinGNN++ framework for constructing a comprehensive interatomic potential for magnetic systems. SpinGNN++ integrates spin equivariant neural network with explicit spin-lattice terms, including Heisenberg, Dzyaloshinskii-Moriya, Kitaev, single-ion anisotropy, and biquadratic interactions. SpinGNN++ identifies a new ferrimagnetic state as the ground magnetic state for monolayer CrTe2.
  arXiv  Detail & Related papers  (2022-11-21T12:25:58Z)
• Visualizing spinon Fermi surfaces with time-dependent spectroscopy [62.997667081978825]
  We propose applying time-dependent photo-emission spectroscopy, an established tool in solid state systems, in cold atom quantum simulators. We show in exact diagonalization simulations of the one-dimensional $t-J$ model that the spinons start to populate previously unoccupied states in an effective band structure. The dependence of the spectral function on the time after the pump pulse reveals collective interactions among spinons.
  arXiv  Detail & Related papers  (2021-05-27T18:00:02Z)
• Ab initio Ultrafast Spin Dynamics in Solids [0.7874708385247353]
  We present a first-principles real-time density-matrix approach based on Lindblad dynamics to simulate ultrafast spin dynamics for general solid-state systems. We find that the electron-electron scattering is negligible at room temperature but becomes dominant at low temperatures for spin relaxation in n-type GaAs.
  arXiv  Detail & Related papers  (2020-12-16T02:49:47Z)
• Parsimonious neural networks learn interpretable physical laws [77.34726150561087]
  We propose parsimonious neural networks (PNNs) that combine neural networks with evolutionary optimization to find models that balance accuracy with parsimony. The power and versatility of the approach is demonstrated by developing models for classical mechanics and to predict the melting temperature of materials from fundamental properties.
  arXiv  Detail & Related papers  (2020-05-08T16:15:47Z)
• Phase Detection with Neural Networks: Interpreting the Black Box [58.720142291102135]
  Neural networks (NNs) usually hinder any insight into the reasoning behind their predictions. We demonstrate how influence functions can unravel the black box of NN when trained to predict the phases of the one-dimensional extended spinless Fermi-Hubbard model at half-filling.
  arXiv  Detail & Related papers  (2020-04-09T17:45:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.