How Can Quantum Deep Learning Improve Large Language Models?

• URL: http://arxiv.org/abs/2509.16244v1
• Date: Wed, 17 Sep 2025 08:18:58 GMT
• Title: How Can Quantum Deep Learning Improve Large Language Models?
• Authors: Emily Jimin Roh, Hyojun Ahn, Samuel Yen-Chi Chen, Soohyun Park, Joongheon Kim,
• Abstract summary: Full fine-tuning achieves strong performance but imposes prohibitive computational and memory costs.<n>Low-rank adaptation (LoRA), Prefix tuning, and sparse low-rank adaptation (SoRA) address this issue by reducing trainable parameters while maintaining competitive accuracy.<n>Recent advances in quantum deep learning introduce novel opportunities through quantum-inspired encoding and parameterized quantum circuits.
• Score: 24.83900854986565
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The rapid progress of large language models (LLMs) has transformed natural language processing, yet the challenge of efficient adaptation remains unresolved. Full fine-tuning achieves strong performance but imposes prohibitive computational and memory costs. Parameter-efficient fine-tuning (PEFT) strategies, such as low-rank adaptation (LoRA), Prefix tuning, and sparse low-rank adaptation (SoRA), address this issue by reducing trainable parameters while maintaining competitive accuracy. However, these methods often encounter limitations in scalability, stability, and generalization across diverse tasks. Recent advances in quantum deep learning introduce novel opportunities through quantum-inspired encoding and parameterized quantum circuits (PQCs). In particular, the quantum-amplitude embedded adaptation (QAA) framework demonstrates expressive model updates with minimal overhead. This paper presents a systematic survey and comparative analysis of conventional PEFT methods and QAA. The analysis demonstrates trade-offs in convergence, efficiency, and representational capacity, while providing insight into the potential of quantum approaches for future LLM adaptation.

Related papers

• AQER: a scalable and efficient data loader for digital quantum computers [62.40228216126285]
  We develop AQER, a scalable AQL method that constructs the loading circuit by systematically reducing entanglement in target states.<n>We conduct systematic experiments to evaluate the effectiveness of AQER, using synthetic datasets, classical image and language datasets, and a quantum many-body state datasets with up to 50 qubits.
  arXiv  Detail & Related papers  (2026-02-02T14:39:42Z)
• Continual Quantum Architecture Search with Tensor-Train Encoding: Theory and Applications to Signal Processing [68.35481158940401]
  CL-QAS is a continual quantum architecture search framework.<n>It mitigates challenges of costly encoding amplitude and forgetting in variational quantum circuits.<n>It achieves controllable robustness expressivity, sample-efficient generalization, and smooth convergence without barren plateaus.
  arXiv  Detail & Related papers  (2026-01-10T02:36:03Z)
• Superpositional Gradient Descent: Harnessing Quantum Principles for Model Training [0.0]
  Superpositional Gradient Descent (SGD) is a novel linking gradient updates with quantum superposition by injecting quantum circuit perturbations.<n>We present a mathematical framework and implement hybrid quantum-classical circuits in PyTorch and Qiskit.
  arXiv  Detail & Related papers  (2025-11-01T16:37:55Z)
• ZeroQAT: Your Quantization-aware Training but Efficient [53.25965863436039]
  Quantization is an effective technique to reduce the deployment cost of large language models (LLMs)<n>Existing low-bit PTQ methods suffer from accuracy degradation because their layer-wise optimization introduces cumulative error propagation and misalignment between local reconstruction objectives and downstream performance.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• TensoMeta-VQC: A Tensor-Train-Guided Meta-Learning Framework for Robust and Scalable Variational Quantum Computing [60.996803677584424]
  TensoMeta-VQC is a novel tensor-train (TT)-guided meta-learning framework designed to improve the robustness and scalability of VQC significantly.<n>Our framework fully delegates the generation of quantum circuit parameters to a classical TT network, effectively decoupling optimization from quantum hardware.
  arXiv  Detail & Related papers  (2025-08-01T23:37:55Z)
• Quantum-Enhanced LLM Efficient Fine Tuning [25.45732471566526]
  Low-Rank Adaptation (LoRA) enables efficient fine-tuning of pre-trained language models through low-rank matrix approximation.<n>We propose Quantum Hybrid Adaptation (QTHA), a parameter-efficient fine-tuning method that integrates a quantum neural network with a tensor network.<n> Experiments demonstrate that QTHA achieves performance comparable to or surpassing LoRA in-efficient fine-tuning.
  arXiv  Detail & Related papers  (2025-03-17T03:59:26Z)
• RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models [53.571195477043496]
  We propose an algorithm named Rotated Straight-Through-Estimator (RoSTE)<n>RoSTE combines quantization-aware supervised fine-tuning (QA-SFT) with an adaptive rotation strategy to reduce activation outliers.<n>Our findings reveal that the prediction error is directly proportional to the quantization error of the converged weights, which can be effectively managed through an optimized rotation configuration.
  arXiv  Detail & Related papers  (2025-02-13T06:44:33Z)
• Q-MAML: Quantum Model-Agnostic Meta-Learning for Variational Quantum Algorithms [4.525216077859531]
  We introduce a new framework for optimizing parameterized quantum circuits (PQCs) that employs a classical, inspired by Model-Agnostic Meta-Learning (MAML) technique.<n>Our framework features a classical neural network, called Learner, which interacts with a PQC using the output of Learner as an initial parameter.<n>In the adaptation phase, the framework requires only a few PQC updates to converge to a more accurate value, while the learner remains unchanged.
  arXiv  Detail & Related papers  (2025-01-10T12:07:00Z)
• Leveraging Pre-Trained Neural Networks to Enhance Machine Learning with Variational Quantum Circuits [48.33631905972908]
  We introduce an innovative approach that utilizes pre-trained neural networks to enhance Variational Quantum Circuits (VQC) This technique effectively separates approximation error from qubit count and removes the need for restrictive conditions. Our results extend to applications such as human genome analysis, demonstrating the broad applicability of our approach.
  arXiv  Detail & Related papers  (2024-11-13T12:03:39Z)
• A Quantum Circuit-Based Compression Perspective for Parameter-Efficient Learning [19.178352290785153]
  We introduce Quantum s Adaptation (QPA) in the framework of quantum parameter generation.<n>QPA integrates QNNs with a classical multi-layer perceptron mapping model to generate parameters for fine-tuning methods.<n>Using Gemma-2 and GPT-2 as case studies, QPA demonstrates significant parameter reduction for parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-10-13T14:09:29Z)
• QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation [8.376592479884042]
  QuanTA is a novel, easy-to-implement, fine-tuning method with no inference overhead for large-scale pre-trained language models. We show that QuanTA significantly enhances commonsense reasoning, arithmetic reasoning, and scalability compared to traditional methods.
  arXiv  Detail & Related papers  (2024-05-31T18:47:30Z)
• PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models [52.09865918265002]
  We propose a novel quantize before fine-tuning'' framework, PreQuant. PreQuant is compatible with various quantization strategies, with outlier-aware fine-tuning incorporated to correct the induced quantization error. We demonstrate the effectiveness of PreQuant on the GLUE benchmark using BERT, RoBERTa, and T5.
  arXiv  Detail & Related papers  (2023-05-30T08:41:33Z)
• Pre-training Tensor-Train Networks Facilitates Machine Learning with Variational Quantum Circuits [70.97518416003358]
  Variational quantum circuits (VQCs) hold promise for quantum machine learning on noisy intermediate-scale quantum (NISQ) devices. While tensor-train networks (TTNs) can enhance VQC representation and generalization, the resulting hybrid model, TTN-VQC, faces optimization challenges due to the Polyak-Lojasiewicz (PL) condition. To mitigate this challenge, we introduce Pre+TTN-VQC, a pre-trained TTN model combined with a VQC.
  arXiv  Detail & Related papers  (2023-05-18T03:08:18Z)
• FLIP: A flexible initializer for arbitrarily-sized parametrized quantum circuits [105.54048699217668]
  We propose a FLexible Initializer for arbitrarily-sized Parametrized quantum circuits. FLIP can be applied to any family of PQCs, and instead of relying on a generic set of initial parameters, it is tailored to learn the structure of successful parameters. We illustrate the advantage of using FLIP in three scenarios: a family of problems with proven barren plateaus, PQC training to solve max-cut problem instances, and PQC training for finding the ground state energies of 1D Fermi-Hubbard models.
  arXiv  Detail & Related papers  (2021-03-15T17:38:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.