Short Quantum Circuits in Reinforcement Learning Policies for the Vehicle Routing Problem

• URL: http://arxiv.org/abs/2109.07498v1
• Date: Wed, 15 Sep 2021 18:02:17 GMT
• Title: Short Quantum Circuits in Reinforcement Learning Policies for the Vehicle Routing Problem
• Authors: Fabio Sanches, Sean Weinberg, Takanori Ide, Kazumitsu Kamiya
• Abstract summary: We show that simple quantum circuits can be used in place of classical attention head layers while maintaining performance. We regard our model as a prototype that can be scaled up and as an avenue for further study on the role of quantum computing in reinforcement learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Quantum computing and machine learning have potential for symbiosis. However, in addition to the hardware limitations from current devices, there are still basic issues that must be addressed before quantum circuits can usefully incorporate with current machine learning tasks. We report a new strategy for such an integration in the context of attention models used for reinforcement learning. Agents that implement attention mechanisms have successfully been applied to certain cases of combinatorial routing problems by first encoding nodes on a graph and then sequentially decoding nodes until a route is selected. We demonstrate that simple quantum circuits can used in place of classical attention head layers while maintaining performance. Our method modifies the networks used in [1] by replacing key and query vectors for every node with quantum states that are entangled before being measured. The resulting hybrid classical-quantum agent is tested in the context of vehicle routing problems where its performance is competitive with the original classical approach. We regard our model as a prototype that can be scaled up and as an avenue for further study on the role of quantum computing in reinforcement learning.

Related papers

• Hybrid Quantum-Classical Machine Learning with String Diagrams [49.1574468325115]
  This paper develops a formal framework for describing hybrid algorithms in terms of string diagrams. A notable feature of our string diagrams is the use of functor boxes, which correspond to a quantum-classical interfaces.
  arXiv  Detail & Related papers  (2024-07-04T06:37:16Z)
• A Quantum-Classical Collaborative Training Architecture Based on Quantum State Fidelity [50.387179833629254]
  We introduce a collaborative classical-quantum architecture called co-TenQu. Co-TenQu enhances a classical deep neural network by up to 41.72% in a fair setting. It outperforms other quantum-based methods by up to 1.9 times and achieves similar accuracy while utilizing 70.59% fewer qubits.
  arXiv  Detail & Related papers  (2024-02-23T14:09:41Z)
• Quantum Deep Hedging [10.243020478772056]
  We look at the problem of hedging where deep reinforcement learning offers a powerful framework for real markets. We develop quantum reinforcement learning methods based on policy-search and distributional actor-critic algorithms. We successfully implement the proposed models on a trapped-ion quantum processor.
  arXiv  Detail & Related papers  (2023-03-29T10:42:50Z)
• Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification [83.20479832949069]
  Quantum neural networks (QNNs) have become an important tool for understanding the physical world, but their advantages and limitations are not fully understood. Here we investigate the problem-dependent power of QCs on multi-class classification tasks. Our work sheds light on the problem-dependent power of QNNs and offers a practical tool for evaluating their potential merit.
  arXiv  Detail & Related papers  (2022-12-29T10:46:40Z)
• Quantum Neural Networks for a Supply Chain Logistics Application [0.0]
  We investigate one such hybrid algorithm on a problem of substantial importance: vehicle routing for supply chain logistics with multiple trucks and complex demand structure. We use reinforcement learning with neural networks with embedded quantum circuits. We find results comparable to human truck assignment.
  arXiv  Detail & Related papers  (2022-11-30T15:35:53Z)
• A didactic approach to quantum machine learning with a single qubit [68.8204255655161]
  We focus on the case of learning with a single qubit, using data re-uploading techniques. We implement the different proposed formulations in toy and real-world datasets using the qiskit quantum computing SDK.
  arXiv  Detail & Related papers  (2022-11-23T18:25:32Z)
• QuanGCN: Noise-Adaptive Training for Robust Quantum Graph Convolutional Networks [124.7972093110732]
  We propose quantum graph convolutional networks (QuanGCN), which learns the local message passing among nodes with the sequence of crossing-gate quantum operations. To mitigate the inherent noises from modern quantum devices, we apply sparse constraint to sparsify the nodes' connections. Our QuanGCN is functionally comparable or even superior than the classical algorithms on several benchmark graph datasets.
  arXiv  Detail & Related papers  (2022-11-09T21:43:16Z)
• Optimizing Tensor Network Contraction Using Reinforcement Learning [86.05566365115729]
  We propose a Reinforcement Learning (RL) approach combined with Graph Neural Networks (GNN) to address the contraction ordering problem. The problem is extremely challenging due to the huge search space, the heavy-tailed reward distribution, and the challenging credit assignment. We show how a carefully implemented RL-agent that uses a GNN as the basic policy construct can address these challenges.
  arXiv  Detail & Related papers  (2022-04-18T21:45:13Z)
• Embedding Learning in Hybrid Quantum-Classical Neural Networks [8.029801398363261]
  We propose a quantum few-shot embedding learning paradigm, which learns embeddings useful for training downstream quantum machine learning tasks. We identify the circuit bypass problem in hybrid neural networks, where learned classical parameters do not utilize the Hilbert space efficiently. We observe that the few-shot learned embeddings generalize to unseen classes and suffer less from the circuit bypass problem compared with other approaches.
  arXiv  Detail & Related papers  (2022-04-09T21:16:20Z)
• Quantum Deep Reinforcement Learning for Robot Navigation Tasks [2.6999000177990924]
  We show that quantum circuits in hybrid quantum-classic reinforcement learning setups are capable of learning optimal policies in multiple robotic navigation scenarios. We find that the employed quantum circuits outperform the classical neural network baselines when equating for the number of trainable parameters.
  arXiv  Detail & Related papers  (2022-02-24T16:26:23Z)
• Variational Quantum Soft Actor-Critic [1.90365714903665]
  We develop a quantum reinforcement learning algorithm based on soft actor-critic -- one of the state-of-the-art methods for continuous control. We show that this quantum version of soft actor-critic is comparable with the original soft actor-critic, using much less adjustable parameters.
  arXiv  Detail & Related papers  (2021-12-20T06:31:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.