Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networks

• URL: http://arxiv.org/abs/2306.05850v1
• Date: Fri, 9 Jun 2023 12:31:59 GMT
• Title: Deterministic equivalent of the Conjugate Kernel matrix associated to Artificial Neural Networks
• Authors: Cl\'ement Chouard (IMT)
• Abstract summary: We show that the empirical spectral distribution of the Conjugate Kernel converges to a deterministic limit. More precisely we obtain a deterministic equivalent for its Stieltjes transform and its resolvent, with quantitative bounds involving both the dimension and the spectral parameter.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the Conjugate Kernel associated to a multi-layer linear-width feed-forward neural network with random weights, biases and data. We show that the empirical spectral distribution of the Conjugate Kernel converges to a deterministic limit. More precisely we obtain a deterministic equivalent for its Stieltjes transform and its resolvent, with quantitative bounds involving both the dimension and the spectral parameter. The limiting equivalent objects are described by iterating free convolution of measures and classical matrix operations involving the parameters of the model.

Related papers

• Simulating NMR Spectra with a Quantum Computer [49.1574468325115]
  This paper provides a formalization of the complete procedure of the simulation of a spin system's NMR spectrum. We also explain how to diagonalize the Hamiltonian matrix with a quantum computer, thus enhancing the overall process's performance.
  arXiv  Detail & Related papers  (2024-10-28T08:43:40Z)
• Spiral flow of quantum quartic oscillator with energy cutoff [0.0]
  The cutoff dependence of the corrected matrices is found to be described by a spiral motion of a three-dimensional vector. This foundational combination of a limit-cycle and a floating fixed-point behaviors warrants further study.
  arXiv  Detail & Related papers  (2024-04-26T14:34:05Z)
• Dilute neutron star matter from neural-network quantum states [58.720142291102135]
  Low-density neutron matter is characterized by the formation of Cooper pairs and the onset of superfluidity. We model this density regime by capitalizing on the expressivity of the hidden-nucleon neural-network quantum states combined with variational Monte Carlo and reconfiguration techniques.
  arXiv  Detail & Related papers  (2022-12-08T17:55:25Z)
• Quantitative deterministic equivalent of sample covariance matrices with a general dependence structure [0.0]
  We prove quantitative bounds involving both the dimensions and the spectral parameter, in particular allowing it to get closer to the real positive semi-line. As applications, we obtain a new bound for the convergence in Kolmogorov distance of the empirical spectral distributions of these general models.
  arXiv  Detail & Related papers  (2022-11-23T15:50:31Z)
• On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels [24.99551134153653]
  We study the properties of various over-parametrized convolutional neural architectures through their respective Gaussian process and tangent neural kernels. We show that the eigenvalues decay hierarchically, quantify the rate of decay, and derive measures that reflect the composition of hierarchical features in these networks.
  arXiv  Detail & Related papers  (2022-03-17T11:23:18Z)
• Entanglement Entropy of Non-Hermitian Free Fermions [59.54862183456067]
  We study the entanglement properties of non-Hermitian free fermionic models with translation symmetry. Our results show that the entanglement entropy has a logarithmic correction to the area law in both one-dimensional and two-dimensional systems.
  arXiv  Detail & Related papers  (2021-05-20T14:46:09Z)
• Hilbert-space geometry of random-matrix eigenstates [55.41644538483948]
  We discuss the Hilbert-space geometry of eigenstates of parameter-dependent random-matrix ensembles. Our results give the exact joint distribution function of the Fubini-Study metric and the Berry curvature. We compare our results to numerical simulations of random-matrix ensembles as well as electrons in a random magnetic field.
  arXiv  Detail & Related papers  (2020-11-06T19:00:07Z)
• The Convergence Indicator: Improved and completely characterized parameter bounds for actual convergence of Particle Swarm Optimization [68.8204255655161]
  We introduce a new convergence indicator that can be used to calculate whether the particles will finally converge to a single point or diverge. Using this convergence indicator we provide the actual bounds completely characterizing parameter regions that lead to a converging swarm.
  arXiv  Detail & Related papers  (2020-06-06T19:08:05Z)
• Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks [22.57374777395746]
  We study the eigenvalue of the Conjugate Neural Kernel and Tangent Kernel associated to feedforward neural networks. We show that the eigenvalue distributions of the CK and NTK converge to deterministic limits.
  arXiv  Detail & Related papers  (2020-05-25T01:11:49Z)
• Operator-algebraic renormalization and wavelets [62.997667081978825]
  We construct the continuum free field as the scaling limit of Hamiltonian lattice systems using wavelet theory. A renormalization group step is determined by the scaling equation identifying lattice observables with the continuum field smeared by compactly supported wavelets.
  arXiv  Detail & Related papers  (2020-02-04T18:04:51Z)
• Variational-Correlations Approach to Quantum Many-body Problems [1.9336815376402714]
  We investigate an approach for studying the ground state of a quantum many-body Hamiltonian. The challenge set by the exponentially-large Hilbert space is circumvented by approximating the positivity of the density matrix. We demonstrate the ability of this approach to produce long-range correlations, and a ground-state energy that converges to the exact result.
  arXiv  Detail & Related papers  (2020-01-17T19:52:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.