Neural annealing and visualization of autoregressive neural networks in the Newman-Moore model

• URL: http://arxiv.org/abs/2204.11272v1
• Date: Sun, 24 Apr 2022 13:15:28 GMT
• Title: Neural annealing and visualization of autoregressive neural networks in the Newman-Moore model
• Authors: Estelle M. Inack, Stewart Morawetz and Roger G. Melko
• Abstract summary: We show that glassy dynamics exhibited by the Newman-Moore model likely manifests itself through trainability issues and mode collapse in the optimization landscape. These findings indicate that the glassy dynamics exhibited by the Newman-Moore model caused by the presence of fracton excitations in the configurational space likely manifests itself through trainability issues and mode collapse in the optimization landscape.
• Score: 0.45119235878273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial neural networks have been widely adopted as ansatzes to study classical and quantum systems. However, some notably hard systems such as those exhibiting glassiness and frustration have mainly achieved unsatisfactory results despite their representational power and entanglement content, thus, suggesting a potential conservation of computational complexity in the learning process. We explore this possibility by implementing the neural annealing method with autoregressive neural networks on a model that exhibits glassy and fractal dynamics: the two-dimensional Newman-Moore model on a triangular lattice. We find that the annealing dynamics is globally unstable because of highly chaotic loss landscapes. Furthermore, even when the correct ground state energy is found, the neural network generally cannot find degenerate ground-state configurations due to mode collapse. These findings indicate that the glassy dynamics exhibited by the Newman-Moore model caused by the presence of fracton excitations in the configurational space likely manifests itself through trainability issues and mode collapse in the optimization landscape.

Related papers

• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Certified Neural Approximations of Nonlinear Dynamics [52.79163248326912]
  In safety-critical contexts, the use of neural approximations requires formal bounds on their closeness to the underlying system.<n>We propose a novel, adaptive, and parallelizable verification method based on certified first-order models.
  arXiv  Detail & Related papers  (2025-05-21T13:22:20Z)
• High-order expansion of Neural Ordinary Differential Equations flows [4.4569182855550755]
  We introduce Event Transitions, a framework based on high-order differentials that provides a rigorous mathematical description of neural ODE dynamics on event gradient. Our findings contribute to a deeper theoretical foundation for event-triggered neural differential equations and provide a mathematical construct for explaining complex system dynamics.
  arXiv  Detail & Related papers  (2025-04-02T08:57:34Z)
• Harnessing and modulating chaos to sample from neural generative models [2.048226951354646]
  We show how neural chaos might play a functional role in allowing the brain to learn and sample from generative models. We construct architectures that combine a classic model of neural chaos either with a canonical generative modeling architecture or with energy-based models of neural memory.
  arXiv  Detail & Related papers  (2024-09-26T22:52:26Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Explosive neural networks via higher-order interactions in curved statistical manifolds [43.496401697112695]
  We introduce curved neural networks as a class of prototypical models for studying higher-order phenomena. We show that these curved neural networks implement a self-regulating process that can accelerate memory retrieval.
  arXiv  Detail & Related papers  (2024-08-05T09:10:29Z)
• Exploiting Chaotic Dynamics as Deep Neural Networks [1.9282110216621833]
  We show that the essence of chaos can be found in various state-of-the-art deep neural networks. Our framework presents superior results in terms of accuracy, convergence speed, and efficiency. This study offers a new path for the integration of chaos, which has long been overlooked in information processing.
  arXiv  Detail & Related papers  (2024-05-29T22:03:23Z)
• NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory [70.10550467873499]
  We propose NeuralClothSim, a new quasistatic cloth simulator using thin shells. Our memory-efficient solver operates on a new continuous coordinate-based surface representation called neural deformation fields.
  arXiv  Detail & Related papers  (2023-08-24T17:59:54Z)
• On the Trade-off Between Efficiency and Precision of Neural Abstraction [62.046646433536104]
  Neural abstractions have been recently introduced as formal approximations of complex, nonlinear dynamical models. We employ formal inductive synthesis procedures to generate neural abstractions that result in dynamical models with these semantics.
  arXiv  Detail & Related papers  (2023-07-28T13:22:32Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Learning the ground state of a non-stoquastic quantum Hamiltonian in a rugged neural network landscape [0.0]
  We investigate a class of universal variational wave-functions based on artificial neural networks. In particular, we show that in the present setup the neural network expressivity and Monte Carlo sampling are not primary limiting factors.
  arXiv  Detail & Related papers  (2020-11-23T05:25:47Z)
• Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
  Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
  arXiv  Detail & Related papers  (2020-11-18T18:52:08Z)
• Sobolev training of thermodynamic-informed neural networks for smoothed elasto-plasticity models with level set hardening [0.0]
  We introduce a deep learning framework designed to train smoothed elastoplasticity models with interpretable components. By recasting the yield function as an evolving level set, we introduce a machine learning approach to predict the solutions of the Hamilton-Jacobi equation.
  arXiv  Detail & Related papers  (2020-10-15T22:43:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.