Variational Autoregressive Networks with probability priors

• URL: http://arxiv.org/abs/2605.16020v1
• Date: Fri, 15 May 2026 14:54:30 GMT
• Title: Variational Autoregressive Networks with probability priors
• Abstract summary: Deep learning approaches, such as neural-network-based samplers, face another serious problem: the difficulty of training the models.<n>In this paper, we demonstrate that incorporating physical priors into the model significantly enhances performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Monte Carlo methods are essential across diverse scientific fields, yet their efficiency is frequently hampered by critical slowing down-a sharp increase in autocorrelation times near phase transitions. Although deep learning approaches, such as neural-network-based samplers, have been proposed to alleviate this issue, they face another serious problem: the difficulty of training the models. This difficulty partially stems from the overly general nature of original machine-learning architectures, which often ignore underlying physical symmetries and force networks to relearn them from scratch. In this paper, we demonstrate that incorporating physical priors into the model significantly enhances performance. Building upon existing strategies that integrate spin-spin interactions, we propose a framework that utilizes a prior probability distribution as a starting point for training. Our results for the Ising model, as well as for the Edwards-Anderson spin glass model, suggest that moving away from `blank slate' models in favor of physics-informed priors reduces the training burden and facilitates the simulation of larger system sizes in discrete spin models.

Related papers

• Orthogonal projection-based regularization for efficient model augmentation [2.6071013155805556]
  Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice.<n>These black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system.<n>A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning identification.
  arXiv  Detail & Related papers  (2025-01-10T10:33:13Z)
• Transferable Post-training via Inverse Value Learning [83.75002867411263]
  We propose modeling changes at the logits level during post-training using a separate neural network (i.e., the value network)<n>After training this network on a small base model using demonstrations, this network can be seamlessly integrated with other pre-trained models during inference.<n>We demonstrate that the resulting value network has broad transferability across pre-trained models of different parameter sizes.
  arXiv  Detail & Related papers  (2024-10-28T13:48:43Z)
• Optimizing Dense Feed-Forward Neural Networks [0.0]
  We propose a novel feed-forward neural network constructing method based on pruning and transfer learning. Our approach can compress the number of parameters by more than 70%. We also evaluate the transfer learning level comparing the refined model and the original one training from scratch a neural network.
  arXiv  Detail & Related papers  (2023-12-16T23:23:16Z)
• Learning-based adaption of robotic friction models [50.72489248401199]
  We introduce a novel approach to adapt an existing friction model to new dynamics using as little data as possible.<n>Our method does not rely on data with external load during training, eliminating the need for external torque sensors.
  arXiv  Detail & Related papers  (2023-10-25T14:50:15Z)
• Towards a Better Theoretical Understanding of Independent Subnetwork Training [56.24689348875711]
  We take a closer theoretical look at Independent Subnetwork Training (IST) IST is a recently proposed and highly effective technique for solving the aforementioned problems. We identify fundamental differences between IST and alternative approaches, such as distributed methods with compressed communication.
  arXiv  Detail & Related papers  (2023-06-28T18:14:22Z)
• Cooperative data-driven modeling [44.99833362998488]
  Data-driven modeling in mechanics is evolving rapidly based on recent machine learning advances. New data and models created by different groups become available, opening possibilities for cooperative modeling. Artificial neural networks suffer from catastrophic forgetting, i.e. they forget how to perform an old task when trained on a new one. This hinders cooperation because adapting an existing model for a new task affects the performance on a previous task trained by someone else.
  arXiv  Detail & Related papers  (2022-11-23T14:27:25Z)
• Slimmable Networks for Contrastive Self-supervised Learning [69.9454691873866]
  Self-supervised learning makes significant progress in pre-training large models, but struggles with small models. We introduce another one-stage solution to obtain pre-trained small models without the need for extra teachers. A slimmable network consists of a full network and several weight-sharing sub-networks, which can be pre-trained once to obtain various networks.
  arXiv  Detail & Related papers  (2022-09-30T15:15:05Z)
• Multi-Objective Physics-Guided Recurrent Neural Networks for Identifying Non-Autonomous Dynamical Systems [0.0]
  We propose a physics-guided hybrid approach for modeling non-autonomous systems under control. This is extended by a recurrent neural network and trained using a sophisticated multi-objective strategy. Experiments conducted on real data reveal substantial accuracy improvements by our approach compared to a physics-based model.
  arXiv  Detail & Related papers  (2022-04-27T14:33:02Z)
• Symplectic Momentum Neural Networks -- Using Discrete Variational Mechanics as a prior in Deep Learning [7.090165638014331]
  This paper introduces Sympic Momentum Networks (SyMo) as models from a discrete formulation of mechanics for non-separable mechanical systems. We show that such combination not only allows these models tol earn from limited data but also provides the models with the capability of preserving the symplectic form and show better long-term behaviour.
  arXiv  Detail & Related papers  (2022-01-20T16:33:19Z)
• Revisit Geophysical Imaging in A New View of Physics-informed Generative Adversarial Learning [2.12121796606941]
  Full waveform inversion produces high-resolution subsurface models. FWI with least-squares function suffers from many drawbacks such as the local-minima problem. Recent works relying on partial differential equations and neural networks show promising performance for two-dimensional FWI. We propose an unsupervised learning paradigm that integrates wave equation with a discriminate network to accurately estimate the physically consistent models.
  arXiv  Detail & Related papers  (2021-09-23T15:54:40Z)
• Neural networks with late-phase weights [66.72777753269658]
  We show that the solutions found by SGD can be further improved by ensembling a subset of the weights in late stages of learning. At the end of learning, we obtain back a single model by taking a spatial average in weight space.
  arXiv  Detail & Related papers  (2020-07-25T13:23:37Z)
• Deep Neural Network in Cusp Catastrophe Model [0.0]
  Catastrophe theory was originally proposed to dynamical systems that exhibit sudden shifts in behavior from small changes in input. Here we show how a Catastrophe model can be trained to learn the dynamics of the Cusp Machine models, without really solving for the generating parameters.
  arXiv  Detail & Related papers  (2020-04-06T00:25:41Z)
• Kernel and Rich Regimes in Overparametrized Models [69.40899443842443]
  We show that gradient descent on overparametrized multilayer networks can induce rich implicit biases that are not RKHS norms. We also demonstrate this transition empirically for more complex matrix factorization models and multilayer non-linear networks.
  arXiv  Detail & Related papers  (2020-02-20T15:43:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.