Formalized Hopfield Networks and Boltzmann Machines

• URL: http://arxiv.org/abs/2512.07766v1
• Date: Mon, 08 Dec 2025 17:48:31 GMT
• Title: Formalized Hopfield Networks and Boltzmann Machines
• Authors: Matteo Cipollina, Michail Karatarakis, Freek Wiedijk,
• Abstract summary: We present a Lean formalization of neural networks, covering both deterministic and deterministic models.<n>We first formalize Hopfield networks, recurrent networks that store patterns as stable states.<n>We then consider networks, where updates are probabilistic and convergence is to a stationary distribution.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural networks are widely used, yet their analysis and verification remain challenging. In this work, we present a Lean 4 formalization of neural networks, covering both deterministic and stochastic models. We first formalize Hopfield networks, recurrent networks that store patterns as stable states. We prove convergence and the correctness of Hebbian learning, a training rule that updates network parameters to encode patterns, here limited to the case of pairwise-orthogonal patterns. We then consider stochastic networks, where updates are probabilistic and convergence is to a stationary distribution. As a canonical example, we formalize the dynamics of Boltzmann machines and prove their ergodicity, showing convergence to a unique stationary distribution using a new formalization of the Perron-Frobenius theorem.

Related papers

• From Overfitting to Reliability: Introducing the Hierarchical Approximate Bayesian Neural Network [3.632251954989679]
  HABNN is a novel approach that uses a Gaussian-inverse-Wishart distribution as a hyperprior of the network's weights.<n>Results indicate that HABNN not only matches but often outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2025-12-15T09:08:42Z)
• Improving Equivariant Networks with Probabilistic Symmetry Breaking [9.164167226137664]
  Equivariant networks encode known symmetries into neural networks, often enhancing generalizations.<n>This poses an important problem, both (1) for prediction tasks on domains where self-symmetries are common, and (2) for generative models, which must break symmetries in order to reconstruct from highly symmetric latent spaces.<n>We present novel theoretical results that establish sufficient conditions for representing such distributions.
  arXiv  Detail & Related papers  (2025-03-27T21:04:49Z)
• Enhancing lattice kinetic schemes for fluid dynamics with Lattice-Equivariant Neural Networks [79.16635054977068]
  We present a new class of equivariant neural networks, dubbed Lattice-Equivariant Neural Networks (LENNs) Our approach develops within a recently introduced framework aimed at learning neural network-based surrogate models Lattice Boltzmann collision operators. Our work opens towards practical utilization of machine learning-augmented Lattice Boltzmann CFD in real-world simulations.
  arXiv  Detail & Related papers  (2024-05-22T17:23:15Z)
• Neural Network Diffusion [45.851945143942885]
  A diffusion model is trained to synthesize latent representations from random noise.<n>This model then generates new representations, which are passed through the autoencoder's decoder to produce new subsets of high-performing network parameters.
  arXiv  Detail & Related papers  (2024-02-20T16:59:03Z)
• Hebbian Learning from First Principles [0.0]
  We postulating the expression of its Hamiltonian for supervised and unsupervised protocols. We show how Lagrangian constraints within entropy extremization force network's outcomes on neural correlations. Remarks on the exponential Hopfield model (as the limit of dense networks with diverging density) and semi-supervised protocols are also provided.
  arXiv  Detail & Related papers  (2024-01-13T16:19:49Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network [59.79008107609297]
  We propose in this paper to approximate the joint posterior over the structure of a Bayesian Network. We use a single GFlowNet whose sampling policy follows a two-phase process. Since the parameters are included in the posterior distribution, this leaves more flexibility for the local probability models.
  arXiv  Detail & Related papers  (2023-05-30T19:16:44Z)
• Gradient Descent in Neural Networks as Sequential Learning in RKBS [63.011641517977644]
  We construct an exact power-series representation of the neural network in a finite neighborhood of the initial weights. We prove that, regardless of width, the training sequence produced by gradient descent can be exactly replicated by regularized sequential learning.
  arXiv  Detail & Related papers  (2023-02-01T03:18:07Z)
• Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth [92.25666446274188]
  sinusoidal neural networks with activations have been proposed as an alternative to networks with traditional activation functions. We first propose a simplified version of such sinusoidal neural networks, which allows both for easier practical implementation and simpler theoretical analysis. We then analyze the behavior of these networks from the neural tangent kernel perspective and demonstrate that their kernel approximates a low-pass filter with an adjustable bandwidth.
  arXiv  Detail & Related papers  (2022-11-26T07:41:48Z)
• A PAC-Bayes oracle inequality for sparse neural networks [0.0]
  We study the Gibbs posterior distribution for sparse deep neural nets in a nonparametric regression setting. We prove an oracle inequality which shows that the method adapts to the unknown regularity and hierarchical structure of the regression function.
  arXiv  Detail & Related papers  (2022-04-26T15:48:24Z)
• Discovering dynamical features of Hodgkin-Huxley-type model of physiological neuron using artificial neural network [0.0]
  We consider Hodgkin-Huxley-type system with two fast and one slow variables. For these two systems we create artificial neural networks that are able to reproduce their dynamics. For the bistable model it means that the network being trained only on one brunch of the solutions recovers another without seeing it during the training.
  arXiv  Detail & Related papers  (2022-03-26T19:04:19Z)
• Monotone operator equilibrium networks [97.86610752856987]
  We develop a new class of implicit-depth model based on the theory of monotone operators, the Monotone Operator Equilibrium Network (monDEQ) We show the close connection between finding the equilibrium point of an implicit network and solving a form of monotone operator splitting problem. We then develop a parameterization of the network which ensures that all operators remain monotone, which guarantees the existence of a unique equilibrium point.
  arXiv  Detail & Related papers  (2020-06-15T17:57:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.