Sampling two-dimensional spin systems with transformers

• URL: http://arxiv.org/abs/2604.27738v1
• Date: Thu, 30 Apr 2026 11:29:28 GMT
• Title: Sampling two-dimensional spin systems with transformers
• Abstract summary: We propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins.<n>Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 times 180$ spins.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to $180 \times 180$ spins in case of the Ising model. The Effective Sample Size of our sampler is $\sim 20$ times larger than that of the previous state-of-the-art neural sampler when trained for the $128 \times 128$ Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train $64\times 64$ spin systems.

Related papers

• Can Neural Networks Achieve Optimal Computational-statistical Tradeoff? An Analysis on Single-Index Model [53.6316818897326]
  We propose a gradient-based algorithm for training a two-layer neural network in time.<n>We show that our algorithm learns a sparse representation that strongly aligns with the unknown signal $star$.<n>We extend our approach to the setting where $star$ is $k$-sparse for $k = o(sqrtd)$ by introducing a novel weight technique.
  arXiv  Detail & Related papers  (2026-06-13T09:34:39Z)
• Poisson Hyperplane Processes with Rectified Linear Units [2.15746572209356]
  We establish the connection between the Poisson hyperplane processes (PHP) and two-layer ReLU neural networks.<n>We show that the PHP with a Gaussian prior is an alternative probabilistic representation to a two-layer ReLU neural network.<n>Our numerical experiments demonstrate that our proposed method outperforms the classic two-layer ReLU neural network.
  arXiv  Detail & Related papers  (2026-01-09T07:12:50Z)
• ReSplat: Learning Recurrent Gaussian Splats [98.14472247275512]
  ReSplat is a feed-forward recurrent Gaussian splatting model that iteratively refines 3D Gaussians without explicitly computing gradients.<n>We introduce a compact reconstruction model that operates in a $16 times$ subsampled space, producing $16 times$ fewer Gaussians than previous per-pixel Gaussian models.<n>Our method achieves state-of-the-art performance while significantly reducing the number of Gaussians and improving the rendering speed.
  arXiv  Detail & Related papers  (2025-10-09T17:59:59Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions.<n>For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling.<n>We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Sampling weights of deep neural networks [1.2370077627846041]
  We introduce a probability distribution, combined with an efficient sampling algorithm, for weights and biases of fully-connected neural networks. In a supervised learning context, no iterative optimization or gradient computations of internal network parameters are needed. We prove that sampled networks are universal approximators.
  arXiv  Detail & Related papers  (2023-06-29T10:13:36Z)
• Simulating first-order phase transition with hierarchical autoregressive networks [0.04588028371034406]
  We apply the Hierarchical Autoregressive Neural (HAN) network sampling algorithm to the two-dimensional $Q$-state Potts model. We quantify the performance of the approach in the vicinity of the first-order phase transition and compare it with that of the Wolff cluster algorithm.
  arXiv  Detail & Related papers  (2022-12-09T16:04:56Z)
• Learning to Learn with Generative Models of Neural Network Checkpoints [71.06722933442956]
  We construct a dataset of neural network checkpoints and train a generative model on the parameters. We find that our approach successfully generates parameters for a wide range of loss prompts. We apply our method to different neural network architectures and tasks in supervised and reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-26T17:59:58Z)
• Hierarchical autoregressive neural networks for statistical systems [0.05156484100374058]
  We propose a hierarchical association of physical degrees of freedom, for instance spins, to neurons which replaces it with the scaling with the linear extent $L$ of the system. We demonstrate our approach on the two-dimensional Ising model by simulating lattices of various sizes up to $128 times 128$ spins, with time benchmarks reaching lattices of size $512 times 512$.
  arXiv  Detail & Related papers  (2022-03-21T13:55:53Z)
• Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics [85.31710759801705]
  Current practice requires expensive computational costs in model training for performance prediction. We propose a novel framework for neural network selection by analyzing the governing dynamics over synaptic connections (edges) during training. Our framework is built on the fact that back-propagation during neural network training is equivalent to the dynamical evolution of synaptic connections.
  arXiv  Detail & Related papers  (2022-01-11T20:53:15Z)
• Exploring Sparse Expert Models and Beyond [51.90860155810848]
  Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost. We propose a simple method called expert prototyping that splits experts into different prototypes and applies $k$ top-$1$ routing. This strategy improves the model quality but maintains constant computational costs, and our further exploration on extremely large-scale models reflects that it is more effective in training larger models.
  arXiv  Detail & Related papers  (2021-05-31T16:12:44Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.