Variational Monte Carlo with Large Patched Transformers

• URL: http://arxiv.org/abs/2306.03921v2
• Date: Sat, 16 Mar 2024 14:43:24 GMT
• Title: Variational Monte Carlo with Large Patched Transformers
• Authors: Kyle Sprague, Stefanie Czischek,
• Abstract summary: Large language models, like transformers, have recently demonstrated immense powers in text and image generation. Here we consider two-dimensional Rydberg atom arrays to demonstrate that transformers reach higher accuracies than conventional recurrent neural networks for variational ground state searches.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models, like transformers, have recently demonstrated immense powers in text and image generation. This success is driven by the ability to capture long-range correlations between elements in a sequence. The same feature makes the transformer a powerful wavefunction ansatz that addresses the challenge of describing correlations in simulations of qubit systems. Here we consider two-dimensional Rydberg atom arrays to demonstrate that transformers reach higher accuracies than conventional recurrent neural networks for variational ground state searches. We further introduce large, patched transformer models, which consider a sequence of large atom patches, and show that this architecture significantly accelerates the simulations. The proposed architectures reconstruct ground states with accuracies beyond state-of-the-art quantum Monte Carlo methods, allowing for the study of large Rydberg systems in different phases of matter and at phase transitions. Our high-accuracy ground state representations at reasonable computational costs promise new insights into general large-scale quantum many-body systems.

Related papers

• Physics-informed Transformers for Electronic Quantum States [0.0]
  We propose a modified variational Monte-Carlo framework for neural quantum-state representations. A Transformer is used to parametrize and autoregressively sample the corrections to the reference state. This work paves the way for more efficient and interpretable neural quantum-state representations.
  arXiv  Detail & Related papers  (2024-12-16T19:00:00Z)
• Learning with SASQuaTCh: a Novel Variational Quantum Transformer Architecture with Kernel-Based Self-Attention [0.464982780843177]
  We present a variational quantum circuit architecture named Self-Attention Sequential Quantum Transformer Channel (SASQuaT) Our approach leverages recent insights from kernel-based operator learning in the context of predicting vision transformer network using simple gate operations and a set of multi-dimensional quantum Fourier transforms. To validate our approach, we consider image classification tasks in simulation and with hardware, where with only 9 qubits and a handful of parameters we are able to simultaneously embed and classify a grayscale image of handwritten digits with high accuracy.
  arXiv  Detail & Related papers  (2024-03-21T18:00:04Z)
• Repeat After Me: Transformers are Better than State Space Models at Copying [53.47717661441142]
  We show that while generalized state space models are promising in terms of inference-time efficiency, they are limited compared to transformer models on tasks that require copying from the input context.
  arXiv  Detail & Related papers  (2024-02-01T21:44:11Z)
• Coherent-State Ladder Time-Dependent Variational Principle for Open Quantum Systems [0.0]
  We present a new paradigm for the dynamical simulation of interacting bosonic systems. The method relies on a variational ansatz for the $n$-boson density matrix, in terms of a superposition of photon-added coherent states. We test our method on several examples, demonstrating its potential application to the predictive simulation of interacting bosonic systems and cat qubits.
  arXiv  Detail & Related papers  (2023-06-23T18:00:00Z)
• Onset of scrambling as a dynamical transition in tunable-range quantum circuits [0.0]
  We identify a dynamical transition marking the onset of scrambling in quantum circuits with different levels of long-range connectivity. We show that as a function of the interaction range for circuits of different structures, the tripartite mutual information exhibits a scaling collapse. In addition to systems with conventional power-law interactions, we identify the same phenomenon in deterministic, sparse circuits.
  arXiv  Detail & Related papers  (2023-04-19T17:37:10Z)
• Transformer variational wave functions for frustrated quantum spin systems [0.0]
  We propose an adaptation of the ViT architecture with complex parameters to define a new class of variational neural-network states. The success of the ViT wave function relies on mixing both local and global operations.
  arXiv  Detail & Related papers  (2022-11-10T11:56:44Z)
• Mega: Moving Average Equipped Gated Attention [150.3124713793503]
  Mega is a simple, theoretically grounded, single-head gated attention mechanism equipped with (exponential) moving average. We show that Mega achieves significant improvements over other sequence models, including variants of Transformers and recent state space models.
  arXiv  Detail & Related papers  (2022-09-21T20:52:17Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)
• Neural-Network Quantum States for Periodic Systems in Continuous Space [66.03977113919439]
  We introduce a family of neural quantum states for the simulation of strongly interacting systems in the presence of periodicity. For one-dimensional systems we find very precise estimations of the ground-state energies and the radial distribution functions of the particles. In two dimensions we obtain good estimations of the ground-state energies, comparable to results obtained from more conventional methods.
  arXiv  Detail & Related papers  (2021-12-22T15:27:30Z)
• Transformers Solve the Limited Receptive Field for Monocular Depth Prediction [82.90445525977904]
  We propose TransDepth, an architecture which benefits from both convolutional neural networks and transformers. This is the first paper which applies transformers into pixel-wise prediction problems involving continuous labels.
  arXiv  Detail & Related papers  (2021-03-22T18:00:13Z)
• Quantum Markov Chain Monte Carlo with Digital Dissipative Dynamics on Quantum Computers [52.77024349608834]
  We develop a digital quantum algorithm that simulates interaction with an environment using a small number of ancilla qubits. We evaluate the algorithm by simulating thermal states of the transverse Ising model.
  arXiv  Detail & Related papers  (2021-03-04T18:21:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.