Quantum-inspired Embeddings Projection and Similarity Metrics for Representation Learning

• URL: http://arxiv.org/abs/2501.04591v1
• Date: Wed, 08 Jan 2025 16:11:31 GMT
• Title: Quantum-inspired Embeddings Projection and Similarity Metrics for Representation Learning
• Authors: Ivan Kankeu, Stefan Gerd Fritsch, Gunnar Schönhoff, Elie Mounzer, Paul Lukowicz, Maximilian Kiefer-Emmanouilidis,
• Abstract summary: A core component of representation learning systems is the projection head, which maps the original embeddings into different, often compressed spaces. We propose a quantum-inspired projection head that includes a corresponding quantum-inspired similarity metric. Specifically, we map classical embeddings onto quantum states in Hilbert space and introduce a quantum circuit-based projection head to reduce embedding dimensionality.
• Score: 1.9946021995522714
• License:
• Abstract: Over the last decade, representation learning, which embeds complex information extracted from large amounts of data into dense vector spaces, has emerged as a key technique in machine learning. Among other applications, it has been a key building block for large language models and advanced computer vision systems based on contrastive learning. A core component of representation learning systems is the projection head, which maps the original embeddings into different, often compressed spaces, while preserving the similarity relationship between vectors. In this paper, we propose a quantum-inspired projection head that includes a corresponding quantum-inspired similarity metric. Specifically, we map classical embeddings onto quantum states in Hilbert space and introduce a quantum circuit-based projection head to reduce embedding dimensionality. To evaluate the effectiveness of this approach, we extended the BERT language model by integrating our projection head for embedding compression. We compared the performance of embeddings, which were compressed using our quantum-inspired projection head, with those compressed using a classical projection head on information retrieval tasks using the TREC 2019 and TREC 2020 Deep Learning benchmarks. The results demonstrate that our quantum-inspired method achieves competitive performance relative to the classical method while utilizing 32 times fewer parameters. Furthermore, when trained from scratch, it notably excels, particularly on smaller datasets. This work not only highlights the effectiveness of the quantum-inspired approach but also emphasizes the utility of efficient, ad hoc low-entanglement circuit simulations within neural networks as a powerful quantum-inspired technique.

Related papers

• Quantum Transfer Learning for MNIST Classification Using a Hybrid Quantum-Classical Approach [0.0]
  This research explores the integration of quantum computing with classical machine learning for image classification tasks. We propose a hybrid quantum-classical approach that leverages the strengths of both paradigms. The experimental results indicate that while the hybrid model demonstrates the feasibility of integrating quantum computing with classical techniques, the accuracy of the final model, trained on quantum outcomes, is currently lower than the classical model trained on compressed features.
  arXiv  Detail & Related papers  (2024-08-05T22:16:27Z)
• Neural auto-designer for enhanced quantum kernels [59.616404192966016]
  We present a data-driven approach that automates the design of problem-specific quantum feature maps. Our work highlights the substantial role of deep learning in advancing quantum machine learning.
  arXiv  Detail & Related papers  (2024-01-20T03:11:59Z)
• ShadowNet for Data-Centric Quantum System Learning [188.683909185536]
  We propose a data-centric learning paradigm combining the strength of neural-network protocols and classical shadows. Capitalizing on the generalization power of neural networks, this paradigm can be trained offline and excel at predicting previously unseen systems. We present the instantiation of our paradigm in quantum state tomography and direct fidelity estimation tasks and conduct numerical analysis up to 60 qubits.
  arXiv  Detail & Related papers  (2023-08-22T09:11:53Z)
• The Quantum Path Kernel: a Generalized Quantum Neural Tangent Kernel for Deep Quantum Machine Learning [52.77024349608834]
  Building a quantum analog of classical deep neural networks represents a fundamental challenge in quantum computing. Key issue is how to address the inherent non-linearity of classical deep learning. We introduce the Quantum Path Kernel, a formulation of quantum machine learning capable of replicating those aspects of deep machine learning.
  arXiv  Detail & Related papers  (2022-12-22T16:06:24Z)
• A didactic approach to quantum machine learning with a single qubit [68.8204255655161]
  We focus on the case of learning with a single qubit, using data re-uploading techniques. We implement the different proposed formulations in toy and real-world datasets using the qiskit quantum computing SDK.
  arXiv  Detail & Related papers  (2022-11-23T18:25:32Z)
• Preliminary study on using vector quantization latent spaces for TTS/VC systems with consistent performance [55.10864476206503]
  We investigate the use of quantized vectors to model the latent linguistic embedding. By enforcing different policies over the latent spaces in the training, we are able to obtain a latent linguistic embedding. Our experiments show that the voice cloning system built with vector quantization has only a small degradation in terms of perceptive evaluations.
  arXiv  Detail & Related papers  (2021-06-25T07:51:35Z)
• Tree tensor network classifiers for machine learning: from quantum-inspired to quantum-assisted [0.0]
  We describe a quantum-assisted machine learning (QAML) method in which multivariate data is encoded into quantum states in a Hilbert space whose dimension is exponentially large in the length of the data vector. We present an approach that can be implemented on gate-based quantum computing devices.
  arXiv  Detail & Related papers  (2021-04-06T02:31:48Z)
• Quantum Self-Supervised Learning [22.953284192004034]
  We propose a hybrid quantum-classical neural network architecture for contrastive self-supervised learning. We apply our best quantum model to classify unseen images on the ibmq_paris quantum computer.
  arXiv  Detail & Related papers  (2021-03-26T18:00:00Z)
• Nearest Centroid Classification on a Trapped Ion Quantum Computer [57.5195654107363]
  We design a quantum Nearest Centroid classifier, using techniques for efficiently loading classical data into quantum states and performing distance estimations. We experimentally demonstrate it on a 11-qubit trapped-ion quantum machine, matching the accuracy of classical nearest centroid classifiers for the MNIST handwritten digits dataset and achieving up to 100% accuracy for 8-dimensional synthetic data.
  arXiv  Detail & Related papers  (2020-12-08T01:10:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.