Scalable Physics-Inspired Transformers for Spin Glasses

• URL: http://arxiv.org/abs/2606.22984v2
• Date: Tue, 23 Jun 2026 03:17:22 GMT
• Title: Scalable Physics-Inspired Transformers for Spin Glasses
• Abstract summary: We develop a physics-inspired transformer with interpretable sparse attention and spin-tailored positional embeddings.<n>It achieves up to two orders of magnitude speedup over vanilla variational autoregressive networks.<n>It can resolve full probability distributions, free energies, and overlap statistics across temperatures for Sherrington-Kirkpatrick and 2D or 3D Edwards-Anderson models.
• Score: 10.411943837080921
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient sampling of the Boltzmann distribution in frustrated spin glasses is central to statistical mechanics and combinatorial optimization. Despite advances in machine-learning-based approaches, two issues persist: limited understanding of why variational models fail to benefit from increased scale, unlike the monotonic scaling law of large language models; and high computational cost on large systems that negates advantages over classical sampling methods. Here, we develop a physics-inspired transformer with interpretable sparse attention and spin-tailored positional embeddings to address these challenges. By further leveraging FlashAttention for parallel ancestral sampling, it achieves up to two orders of magnitude speedup over vanilla variational autoregressive networks, enabling neural-network simulations of spin-glass systems to unprecedented sizes on a single GPU. It can resolve full probability distributions, free energies, and overlap statistics across temperatures, for Sherrington-Kirkpatrick and 2D or 3D Edwards-Anderson models, where existing machine-learning methods encounter limitations at certain temperatures. This framework thus establishes a scalable paradigm for frustrated spin-glass systems.

Related papers

• ClothTransformer: Unified Latent-Space Transformers for Scalable Cloth Simulation [28.646171436727233]
  We present ClothTransformer, a framework that reformulates cloth simulation as autoregressive sequence modeling.<n>Existing neural cloth simulators are largely specialized to single scenarios, intrinsically coupled to the mesh discretization.<n>Our approach addresses these limitations through three contributions.
  arXiv  Detail & Related papers  (2026-05-27T02:10:58Z)
• Variational Autoregressive Networks with probability priors [0.0]
  Deep learning approaches, such as neural-network-based samplers, face another serious problem: the difficulty of training the models.<n>In this paper, we demonstrate that incorporating physical priors into the model significantly enhances performance.
  arXiv  Detail & Related papers  (2026-05-15T14:54:30Z)
• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• DiffusionHarmonizer: Bridging Neural Reconstruction and Photorealistic Simulation with Online Diffusion Enhancer [62.18680935878919]
  We introduce DiffusionHarmonizer, an online generative enhancement framework that transforms renderings into temporally consistent outputs.<n>At its core is a single-step temporally-conditioned enhancer capable of running in online simulators on a single GPU.
  arXiv  Detail & Related papers  (2026-02-27T15:35:30Z)
• From Black Hole to Galaxy: Neural Operator: Framework for Accretion and Feedback Dynamics [70.27068115318681]
  We introduce a neural-based ''subgrid black hole'' that learns the small-scale local dynamics and embeds it within direct simulations.<n>Thanks to the great speedup in fine-scale evolution, our approach captures intrinsic variability in accretion-driven feedback, allowing dynamic coupling between the central black hole and galaxy-scale gas.
  arXiv  Detail & Related papers  (2025-12-01T11:47:49Z)
• Quantized Visual Geometry Grounded Transformer [67.15451442018258]
  This paper proposes the first Quantization framework for VGGTs, namely QuantVGGT.<n>We introduce Dual-Smoothed Fine-Grained Quantization, which integrates pre-global Hadamard rotation and post-local channel smoothing.<n>We also design Noise-Filtered Diverse Sampling, which filters outliers via deep-layer statistics.
  arXiv  Detail & Related papers  (2025-09-25T15:17:11Z)
• Enhanced accuracy through ensembling of randomly initialized auto-regressive models for time-dependent PDEs [0.0]
  Autoregressive inference with machine learning models suffer from error accumulation over successive predictions, limiting their long-term accuracy.<n>We propose a deep ensemble framework to address this challenge, where multiple ML surrogate models are trained in parallel and aggregated during inference.<n>We validate the framework on three PDE-driven dynamical systems - stress evolution in heterogeneous microstructures, Gray-Scott reaction-diffusion, and planetary-scale shallow water system.
  arXiv  Detail & Related papers  (2025-07-05T02:25:12Z)
• Optimizing Temperature Distributions for Training Neural Quantum States using Parallel Tempering [0.0]
  We show that temperature optimization can significantly increase the success rate of variational algorithms.<n>We demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network.
  arXiv  Detail & Related papers  (2024-10-30T13:48:35Z)
• Learning Controllable Adaptive Simulation for Multi-resolution Physics [86.8993558124143]
  We introduce Learning controllable Adaptive simulation for Multi-resolution Physics (LAMP) as the first full deep learning-based surrogate model. LAMP consists of a Graph Neural Network (GNN) for learning the forward evolution, and a GNN-based actor-critic for learning the policy of spatial refinement and coarsening. We demonstrate that our LAMP outperforms state-of-the-art deep learning surrogate models, and can adaptively trade-off computation to improve long-term prediction error.
  arXiv  Detail & Related papers  (2023-05-01T23:20:27Z)
• Haar Wavelet based Block Autoregressive Flows for Trajectories [129.37479472754083]
  Prediction of trajectories such as that of pedestrians is crucial to the performance of autonomous agents. We introduce a novel Haar wavelet based block autoregressive model leveraging split couplings. We illustrate the advantages of our approach for generating diverse and accurate trajectories on two real-world datasets.
  arXiv  Detail & Related papers  (2020-09-21T13:57:10Z)
• A Probability Density Theory for Spin-Glass Systems [0.0]
  We develop a continuous probability density theory for spin-glass systems with arbitrary dimensions, interactions, and local fields. We show how our geometrically encodes key physical computational formulation of the spinglass model. We apply our formalism to a number of spin-glass models including the She-Kirkrrington (SK) model, spins on random ErdHos-R'enyi graphs, and restricted Boltzmann machines.
  arXiv  Detail & Related papers  (2020-01-03T18:43:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.