Quantum Large Language Models via Tensor Network Disentanglers

• URL: http://arxiv.org/abs/2410.17397v1
• Date: Tue, 22 Oct 2024 20:12:04 GMT
• Title: Quantum Large Language Models via Tensor Network Disentanglers
• Authors: Borja Aizpurua, Saeed S. Jahromi, Sukhbinder Singh, Roman Orus,
• Abstract summary: We propose a method to enhance the performance of Large Language Models (LLMs) by integrating quantum computing and quantum-inspired techniques. Our approach involves replacing the weight matrices in the Self-Attention and Multi-layer Perceptron layers with a combination of two variational quantum circuits and a quantum-inspired tensor network.
• Score: 0.0
• License:
• Abstract: We propose a method to enhance the performance of Large Language Models (LLMs) by integrating quantum computing and quantum-inspired techniques. Specifically, our approach involves replacing the weight matrices in the Self-Attention and Multi-layer Perceptron layers with a combination of two variational quantum circuits and a quantum-inspired tensor network, such as a Matrix Product Operator (MPO). This substitution enables the reproduction of classical LLM functionality by decomposing weight matrices through the application of tensor network disentanglers and MPOs, leveraging well-established tensor network techniques. By incorporating more complex and deeper quantum circuits, along with increasing the bond dimensions of the MPOs, our method captures additional correlations within the quantum-enhanced LLM, leading to improved accuracy beyond classical models while maintaining low memory overhead.

Related papers

• Quantum-Train with Tensor Network Mapping Model and Distributed Circuit Ansatz [0.8192907805418583]
  Quantum-Train (QT) is a hybrid quantum-classical machine learning framework. It maps quantum state measurements to classical neural network weights. Traditional QT framework employs a multi-layer perceptron (MLP) for this task, but it struggles with scalability and interpretability. We introduce a distributed circuit ansatz designed for large-scale quantum machine learning with multiple small quantum processing unit nodes.
  arXiv  Detail & Related papers  (2024-09-11T03:51:34Z)
• Efficient Learning for Linear Properties of Bounded-Gate Quantum Circuits [63.733312560668274]
  Given a quantum circuit containing d tunable RZ gates and G-d Clifford gates, can a learner perform purely classical inference to efficiently predict its linear properties? We prove that the sample complexity scaling linearly in d is necessary and sufficient to achieve a small prediction error, while the corresponding computational complexity may scale exponentially in d. We devise a kernel-based learning model capable of trading off prediction error and computational complexity, transitioning from exponential to scaling in many practical settings.
  arXiv  Detail & Related papers  (2024-08-22T08:21:28Z)
• Towards Efficient Quantum Hybrid Diffusion Models [68.43405413443175]
  We propose a new methodology to design quantum hybrid diffusion models. We propose two possible hybridization schemes combining quantum computing's superior generalization with classical networks' modularity.
  arXiv  Detail & Related papers  (2024-02-25T16:57:51Z)
• Quantum circuit synthesis with diffusion models [0.6554326244334868]
  We use generative machine learning models, specifically denoising diffusion models (DMs), to facilitate this transformation. We steer the model to produce desired quantum operations within gate-based quantum circuits. We envision DMs as pivotal in quantum circuit synthesis, enhancing both practical applications but also insights into theoretical quantum computation.
  arXiv  Detail & Related papers  (2023-11-03T17:17:08Z)
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning [52.77024349608834]
  Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing. Key issue is how to address the inherent non-linearity of classical deep learning. We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
  arXiv  Detail & Related papers  (2022-12-22T16:06:24Z)
• Quantum Phase Recognition using Quantum Tensor Networks [0.0]
  This paper examines a quantum machine learning approach based on shallow variational ansatz inspired by tensor networks for supervised learning tasks. We are able to reach $geq 98%$ test-set accuracies with both multi-scale entanglement renormalization ansatz (MERA) and tree tensor network (TTN) inspired parametrized quantum circuits.
  arXiv  Detail & Related papers  (2022-12-12T19:29:07Z)
• BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation [116.26521375592759]
  Quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation. Extreme quantization (1-bit weight/1-bit activations) of compactly-designed backbone architectures results in severe performance degeneration. This paper proposes a novel Quantization-Aware Training (QAT) method that can effectively alleviate performance degeneration.
  arXiv  Detail & Related papers  (2022-07-04T13:25:49Z)
• Variational Quantum Optimization with Multi-Basis Encodings [62.72309460291971]
  We introduce a new variational quantum algorithm that benefits from two innovations: multi-basis graph complexity and nonlinear activation functions. Our results in increased optimization performance, two increase in effective landscapes and a reduction in measurement progress.
  arXiv  Detail & Related papers  (2021-06-24T20:16:02Z)
• The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning [84.33745072274942]
  We show how to represent linear and non-linear layers as unitary quantum gates, and interpret the fundamental excitations of the quantum model as particles. On top of opening a new perspective and techniques for studying neural networks, the quantum formulation is well suited for optical quantum computing.
  arXiv  Detail & Related papers  (2021-03-08T17:24:29Z)
• An end-to-end trainable hybrid classical-quantum classifier [0.0]
  We introduce a hybrid model combining a quantum-inspired tensor network and a variational quantum circuit to perform supervised learning tasks. This architecture allows for the classical and quantum parts of the model to be trained simultaneously, providing an end-to-end training framework.
  arXiv  Detail & Related papers  (2021-02-04T05:19:54Z)
• Autoregressive Transformer Neural Network for Simulating Open Quantum Systems via a Probabilistic Formulation [5.668795025564699]
  We present an approach for tackling open quantum system dynamics. We compactly represent quantum states with autoregressive transformer neural networks. Efficient algorithms have been developed to simulate the dynamics of the Liouvillian superoperator.
  arXiv  Detail & Related papers  (2020-09-11T18:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.