Neural annealing and visualization of autoregressive neural networks in the Newman-Moore model

• URL: http://arxiv.org/abs/2204.11272v1
• Date: Sun, 24 Apr 2022 13:15:28 GMT
• Title: Neural annealing and visualization of autoregressive neural networks in the Newman-Moore model
• Authors: Estelle M. Inack, Stewart Morawetz and Roger G. Melko
• Abstract summary: We show that glassy dynamics exhibited by the Newman-Moore model likely manifests itself through trainability issues and mode collapse in the optimization landscape. These findings indicate that the glassy dynamics exhibited by the Newman-Moore model caused by the presence of fracton excitations in the configurational space likely manifests itself through trainability issues and mode collapse in the optimization landscape.
• Score: 0.45119235878273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial neural networks have been widely adopted as ansatzes to study classical and quantum systems. However, some notably hard systems such as those exhibiting glassiness and frustration have mainly achieved unsatisfactory results despite their representational power and entanglement content, thus, suggesting a potential conservation of computational complexity in the learning process. We explore this possibility by implementing the neural annealing method with autoregressive neural networks on a model that exhibits glassy and fractal dynamics: the two-dimensional Newman-Moore model on a triangular lattice. We find that the annealing dynamics is globally unstable because of highly chaotic loss landscapes. Furthermore, even when the correct ground state energy is found, the neural network generally cannot find degenerate ground-state configurations due to mode collapse. These findings indicate that the glassy dynamics exhibited by the Newman-Moore model caused by the presence of fracton excitations in the configurational space likely manifests itself through trainability issues and mode collapse in the optimization landscape.

Related papers

• Exploiting Chaotic Dynamics as Deep Neural Networks [1.9282110216621833]
  We show that the essence of chaos can be found in various state-of-the-art deep neural networks. Our framework presents superior results in terms of accuracy, convergence speed, and efficiency. This study offers a new path for the integration of chaos, which has long been overlooked in information processing.
  arXiv  Detail & Related papers  (2024-05-29T22:03:23Z)
• Learning Dissipative Neural Dynamical Systems [0.8993153817914281]
  In general, imposing dissipativity constraints during neural network training is a hard problem for which no known techniques exist. We show that these two perturbation problems can be solved independently to obtain a neural dynamical model guaranteed to be dissipative.
  arXiv  Detail & Related papers  (2023-09-27T21:25:26Z)
• NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory [70.10550467873499]
  We propose NeuralClothSim, a new quasistatic cloth simulator using thin shells. Our memory-efficient solver operates on a new continuous coordinate-based surface representation called neural deformation fields. NDFs are adaptive: They allocate their capacity to the deformation details and 2) allow surface state queries at arbitrary spatial resolutions without re-training.
  arXiv  Detail & Related papers  (2023-08-24T17:59:54Z)
• On the Trade-off Between Efficiency and Precision of Neural Abstraction [62.046646433536104]
  Neural abstractions have been recently introduced as formal approximations of complex, nonlinear dynamical models. We employ formal inductive synthesis procedures to generate neural abstractions that result in dynamical models with these semantics.
  arXiv  Detail & Related papers  (2023-07-28T13:22:32Z)
• Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks? [18.92828441607381]
  We show that embeddings induce a model that fits observations well but simultaneously has incorrect dynamical behaviours. We propose a simple embedding-free alternative based on parametrising two additive vector-field components.
  arXiv  Detail & Related papers  (2023-05-20T12:41:47Z)
• ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Prediction [82.81767856234956]
  This paper proposes a new learning framework named ConCerNet to improve the trustworthiness of the DNN based dynamics modeling. We show that our method consistently outperforms the baseline neural networks in both coordinate error and conservation metrics.
  arXiv  Detail & Related papers  (2023-02-11T21:07:30Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Learning the ground state of a non-stoquastic quantum Hamiltonian in a rugged neural network landscape [0.0]
  We investigate a class of universal variational wave-functions based on artificial neural networks. In particular, we show that in the present setup the neural network expressivity and Monte Carlo sampling are not primary limiting factors.
  arXiv  Detail & Related papers  (2020-11-23T05:25:47Z)
• Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
  Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
  arXiv  Detail & Related papers  (2020-11-18T18:52:08Z)
• Sobolev training of thermodynamic-informed neural networks for smoothed elasto-plasticity models with level set hardening [0.0]
  We introduce a deep learning framework designed to train smoothed elastoplasticity models with interpretable components. By recasting the yield function as an evolving level set, we introduce a machine learning approach to predict the solutions of the Hamilton-Jacobi equation.
  arXiv  Detail & Related papers  (2020-10-15T22:43:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.