Scaling Autoregressive Models for Lattice Thermodynamics

• URL: http://arxiv.org/abs/2603.14695v1
• Date: Mon, 16 Mar 2026 01:03:59 GMT
• Title: Scaling Autoregressive Models for Lattice Thermodynamics
• Abstract summary: Markov-chain Monte Carlo sampling suffers from slow convergence and critical slowing down near phase transitions.<n>Existing autoregressive models (ARMs) generate configurations in a fixed sequential order and incur high memory and training costs.<n>Here, we develop a framework combining any-order ARMs, which generate configurations flexibly by conditioning on any known subset of lattice sites, with models (MAMs) which approximate the probability of any partial configuration in a single forward pass.
• Score: 5.904448014233719
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting how materials behave under realistic conditions requires understanding the statistical distribution of atomic configurations on crystal lattices, a problem central to alloy design, catalysis, and the study of phase transitions. Traditional Markov-chain Monte Carlo sampling suffers from slow convergence and critical slowing down near phase transitions, motivating the use of generative models that directly learn the thermodynamic distribution. Existing autoregressive models (ARMs), however, generate configurations in a fixed sequential order and incur high memory and training costs, limiting their applicability to realistic systems. Here, we develop a framework combining any-order ARMs, which generate configurations flexibly by conditioning on any known subset of lattice sites, with marginalization models (MAMs), which approximate the probability of any partial configuration in a single forward pass and substantially reduce memory requirements. This combination enables models trained on smaller lattices to be reused for sampling larger systems, while supporting expressive Transformer architectures with lattice-aware positional encodings at manageable computational cost. We demonstrate that Transformer-based any-order MAMs achieve more accurate free energies than multilayer perceptron-based ARMs on both the two-dimensional Ising model and CuAu alloys, faithfully capturing phase transitions and critical behavior. Overall, our framework scales from $10 \times 10$ to $20 \times 20$ Ising systems and from $2 \times 2 \times 4$ to $4 \times 4 \times 8$ CuAu supercells at reduced computational cost compared to conventional sampling methods.

Related papers

• The HALO Engine: $\mathcal{O}(1)$-Step Compilation and Localized String Rupture for Lattice Gauge Theories on Quantum Hardware [0.0]
  We introduce the Hardware-Aware Lattice Optimization (HALO) compiler, which executes global time-evolution steps in an immutable $mathcalO(1)$ circuit depth per Trotter step.<n>We execute the non-perturbative dynamics of a heavily stretched $L=15$ meson string on a 16-qubit superconducting transmon processor.<n>We map the dynamical phase diagram of the mesoscopic lattice, pinpointing the effective confinement phase boundary at precisely $g_c = 1.0$.
  arXiv  Detail & Related papers  (2026-08-14T13:07:47Z)
• Compute-Optimal Is Not Cluster-Optimal: Systems-Aware Scaling for Sparse Mixture-of-Experts [13.835014453826057]
  We develop MOSAIC, which formulates model architecture and systems co-design.<n>We instantiate the framework for sparse Mixture-of-Experts (MoE) language models.<n>We show that, within the calibrated sparsity range, an efficiency-agnostic model-FLOPs budget admits no interior optimal sparsity.
  arXiv  Detail & Related papers  (2026-08-11T07:49:00Z)
• Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulation [0.9668407688201359]
  $$N-Ens is a normalisation-based implicit ensemble that treats each member as a task in a multi-task architecture.<n>Because only normalisation layers are replicated, the mechanism can wrap convolutional and transformer backbones alike.
  arXiv  Detail & Related papers  (2026-07-26T22:06:42Z)
• Unlocking Feature Learning in Gated Delta Networks at Scale [52.705562336559446]
  Training and scaling Large Language Models demand enormous computational resources.<n>We derive the scaling rules for Gated Delta Network.<n> Experiments on language-model pre-training confirm that our configurations enable stable learning-rate transfer.
  arXiv  Detail & Related papers  (2026-06-02T08:45:24Z)
• FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment [20.331469310989956]
  We argue that importance-ordered nested components can be extracted from pretrained models, and selectively activated on the available computational budget.<n>Our approach enables a "train-once, deploy-everywhere" paradigm that offers a graceful trade-off between cost and performance without training from scratch for each budget.
  arXiv  Detail & Related papers  (2026-02-02T19:01:40Z)
• EcoSpa: Efficient Transformer Training with Coupled Sparsity [79.5008521101473]
  Transformers have become the backbone of modern AI, yet their high computational demands pose critical system challenges.<n>We introduce EcoSpa, an efficient structured sparse training method that jointly evaluates and sparsifies coupled weight matrix pairs.
  arXiv  Detail & Related papers  (2025-11-09T11:23:43Z)
• Equilibrium Matching: Generative Modeling with Implicit Energy-Based Models [52.74448905289362]
  EqM is a generative modeling framework built from an equilibrium dynamics perspective.<n>By replacing time-conditional velocities with a unified equilibrium landscape, EqM offers a tighter bridge between flow and energy-based models.
  arXiv  Detail & Related papers  (2025-10-02T17:59:06Z)
• PT$^2$-LLM: Post-Training Ternarization for Large Language Models [52.4629647715623]
  Large Language Models (LLMs) have shown impressive capabilities across diverse tasks, but their large memory and compute demands hinder deployment.<n>We propose PT$2$-LLM, a post-training ternarization framework tailored for LLMs.<n>At its core is an Asymmetric Ternary Quantizer equipped with a two-stage refinement pipeline.
  arXiv  Detail & Related papers  (2025-09-27T03:01:48Z)
• Flow Matching at Scale: A Machine Learning Framework for Efficient Large-Size Sampling of Many-Body Systems [0.0]
  We propose a machine learning framework based on Flow Matching to overcome the scaling limitations of Markov Chain Monte Carlo methods.<n>We demonstrate its capability in the 2D XY model, where a single network, trained only on configurations from a small lattice at sparse temperature points, generates reliable samples for a significantly larger system.
  arXiv  Detail & Related papers  (2025-08-21T07:29:25Z)
• Efficient dataset construction using active learning and uncertainty-aware neural networks for plasma turbulent transport surrogate models [0.0]
  This work demonstrates a proof-of-principle for using uncertainty-aware architectures to construct efficient datasets for surrogate model generation.<n>This strategy was applied again to the plasma turbulent transport problem within tokamak fusion plasmas, specifically the QuaLiKiz quasilinear electrostatic gyrokinetic turbulent transport code.<n>With 45 active learning iterations, moving from a small initial training set of $102$ to a final set of $104$, the resulting models reached a $F_1$ classification performance of 0.8 and a $R2$ regression performance of 0.75 on an independent test set
  arXiv  Detail & Related papers  (2025-07-21T18:15:12Z)
• Minitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning [76.88243649182886]
  Hybrid architectures that combine Attention and State Space Models (SSMs) achieve state-of-the-art accuracy and runtime performance.<n>Recent work has demonstrated that applying compression and distillation to Attention-only models yields smaller, more accurate models at a fraction of the training cost.<n>We introduce a novel group-aware pruning strategy that preserves the structural integrity of SSM blocks and their sequence modeling capabilities.
  arXiv  Detail & Related papers  (2025-04-15T17:26:29Z)
• Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget [53.311109531586844]
  We demonstrate very low-cost training of large-scale T2I diffusion transformer models. We train a 1.16 billion parameter sparse transformer with only $1,890 economical cost and achieve a 12.7 FID in zero-shot generation. We aim to release our end-to-end training pipeline to further democratize the training of large-scale diffusion models on micro-budgets.
  arXiv  Detail & Related papers  (2024-07-22T17:23:28Z)
• TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models [52.454274602380124]
  Diffusion models heavily depend on the time-step $t$ to achieve satisfactory multi-round denoising. We propose a Temporal Feature Maintenance Quantization (TFMQ) framework building upon a Temporal Information Block. Powered by the pioneering block design, we devise temporal information aware reconstruction (TIAR) and finite set calibration (FSC) to align the full-precision temporal features.
  arXiv  Detail & Related papers  (2023-11-27T12:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.