Short sighted deep learning

• URL: http://arxiv.org/abs/2002.02664v1
• Date: Fri, 7 Feb 2020 08:33:07 GMT
• Title: Short sighted deep learning
• Authors: Ellen de Melllo Koch, Anita de Mello Koch, Nicholas Kastanos, Ling Cheng
• Abstract summary: We extend the discussion to the setting of a long range spin lattice. MCMC simulations determine both the critical temperature and scaling dimensions of the system.
• Score: 0.22940141855172028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A theory explaining how deep learning works is yet to be developed. Previous work suggests that deep learning performs a coarse graining, similar in spirit to the renormalization group (RG). This idea has been explored in the setting of a local (nearest neighbor interactions) Ising spin lattice. We extend the discussion to the setting of a long range spin lattice. Markov Chain Monte Carlo (MCMC) simulations determine both the critical temperature and scaling dimensions of the system. The model is used to train both a single RBM (restricted Boltzmann machine) network, as well as a stacked RBM network. Following earlier Ising model studies, the trained weights of a single layer RBM network define a flow of lattice models. In contrast to results for nearest neighbor Ising, the RBM flow for the long ranged model does not converge to the correct values for the spin and energy scaling dimension. Further, correlation functions between visible and hidden nodes exhibit key differences between the stacked RBM and RG flows. The stacked RBM flow appears to move towards low temperatures whereas the RG flow moves towards high temperature. This again differs from results obtained for nearest neighbor Ising.

Related papers

• von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
  We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Active search and coverage using point-cloud reinforcement learning [50.741409008225766]
  This paper presents an end-to-end deep reinforcement learning solution for target search and coverage. We show that deep hierarchical feature learning works for RL and that by using farthest point sampling (FPS) we can reduce the amount of points. We also show that multi-head attention for point-clouds helps to learn the agent faster but converges to the same outcome.
  arXiv  Detail & Related papers  (2023-12-18T18:16:30Z)
• A Deep Dive into the Connections Between the Renormalization Group and Deep Learning in the Ising Model [0.0]
  Renormalization group (RG) is an essential technique in statistical physics and quantum field theory. We develop extensive renormalization techniques for the 1D and 2D Ising model to provide a baseline for comparison. For the 2D Ising model, we successfully generated Ising model samples using the Wolff algorithm, and performed the group flow using a quasi-deterministic method.
  arXiv  Detail & Related papers  (2023-08-21T22:50:54Z)
• Optimal Connectivity through Network Gradients for the Restricted Boltzmann Machine [0.0]
  A fundamental problem is efficiently finding connectivity patterns that improve the learning curve. Recent approaches explicitly include network connections as parameters that must be optimized in the model. This work presents a method to find optimal connectivity patterns for RBMs based on the idea of network gradients.
  arXiv  Detail & Related papers  (2022-09-14T21:09:58Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• Feature extraction of machine learning and phase transition point of Ising model [0.0]
  We study the features extracted by the Restricted Boltzmann Machine (RBM) when it is trained with spin configurations of Ising model at various temperatures. We find that in some cases the flow approaches the phase transition point $T=T_c$ in Ising model.
  arXiv  Detail & Related papers  (2021-11-22T13:04:24Z)
• Repesentation of general spin-$S$ systems using a Restricted Boltzmann Machine with Softmax Regression [0.0]
  We have shown that proposed SRBM technique performs very well and achieves the trial wave function, in a numerically more efficient way. We evaluated the accuracy of our method by studying the spin-1/2 systems with softmax RBM.
  arXiv  Detail & Related papers  (2021-09-22T11:14:18Z)
• What Are Bayesian Neural Network Posteriors Really Like? [63.950151520585024]
  We show that Hamiltonian Monte Carlo can achieve significant performance gains over standard and deep ensembles. We also show that deep distributions are similarly close to HMC as standard SGLD, and closer than standard variational inference.
  arXiv  Detail & Related papers  (2021-04-29T15:38:46Z)
• Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks [61.30171206892684]
  We develop a hybrid of intra- and inter-layer model learning that considers the network as a multi-layer cluster-based structure. MH-FL considers the topology structures among the nodes in the clusters, including local networks formed via device-to-device (D2D) communications. It orchestrates the devices at different network layers in a collaborative/cooperative manner to form local consensus on the model parameters.
  arXiv  Detail & Related papers  (2020-07-18T20:03:07Z)
• Restricted Boltzmann Machine Flows and The Critical Temperature of Ising models [0.0]
  We explore alternative experimental setups for the iterative sampling (flow) from Boltzmann Machines (RBM) This framework has been introduced to explore connections between RBM-based deep neural networks and the Renormalization Group (RG)
  arXiv  Detail & Related papers  (2020-06-17T21:59:00Z)
• Kernel and Rich Regimes in Overparametrized Models [69.40899443842443]
  We show that gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms. We also demonstrate this transition empirically for more complex matrix factorization models and multilayer non-linear networks.
  arXiv  Detail & Related papers  (2020-02-20T15:43:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.