Homology-constrained vector quantization entropy regularizer

• URL: http://arxiv.org/abs/2211.14363v1
• Date: Fri, 25 Nov 2022 20:09:22 GMT
• Title: Homology-constrained vector quantization entropy regularizer
• Authors: Ivan Volkov
• Abstract summary: This paper describes an entropy regularization term for vector quantization (VQ) based on the analysis of persistent homology of the VQ embeddings. We show that homology-constrained regularization is an effective way to increase entropy of the VQ process.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes an entropy regularization term for vector quantization (VQ) based on the analysis of persistent homology of the VQ embeddings. Higher embedding entropy positively correlates with higher codebook utilization, mitigating overfit towards the identity and codebook collapse in VQ-based autoencoders [1]. We show that homology-constrained regularization is an effective way to increase entropy of the VQ process (approximated to input entropy) while preserving the approximated topology in the quantized latent space, averaged over mini batches. This work further explores some patterns of persistent homology diagrams of latents formed by vector quantization. We implement and test the proposed algorithm as a module integrated into a sample VQ-VAE. Linked code repository provides a functioning implementation of the proposed architecture, referred to as homology-constrained vector quantization (HC-VQ) further in this work.

Related papers

• VQ-Flow: Taming Normalizing Flows for Multi-Class Anomaly Detection via Hierarchical Vector Quantization [101.41553763861381]
  This paper explores the potential of normalizing flows in multi-class anomaly detection. We empower the flow models to distinguish different concepts of multi-class normal data in an unsupervised manner, resulting in a novel flow-based unified method, named VQ-Flow. The proposed VQ-Flow advances the state-of-the-art in multi-class anomaly detection within a unified training scheme, yielding the Det./Loc. AUROC of 99.5%/98.3% on MVTec AD.
  arXiv  Detail & Related papers  (2024-09-02T05:01:41Z)
• Quantum walk informed variational algorithm design [0.0]
  We present a theoretical framework for the analysis of amplitude transfer in Quantum Variational Algorithms (QVAs) We develop algorithms for unconstrained and constrained novel optimisations. We show significantly improved convergence over preexisting QVAs.
  arXiv  Detail & Related papers  (2024-06-17T15:04:26Z)
• HyperVQ: MLR-based Vector Quantization in Hyperbolic Space [56.4245885674567]
  We study the use of hyperbolic spaces for vector quantization (HyperVQ) We show that hyperVQ performs comparably in reconstruction and generative tasks while outperforming VQ in discriminative tasks and learning a highly disentangled latent space.
  arXiv  Detail & Related papers  (2024-03-18T03:17:08Z)
• Vector Quantization for Deep-Learning-Based CSI Feedback in Massive MIMO Systems [7.934232975873179]
  This paper presents a finite-rate deep-learning (DL)-based channel state information (CSI) feedback method for massive multiple-input multiple-output (MIMO) systems. The presented method provides a finite-bit representation of the latent vector based on a vector-quantized variational autoencoder (VQ-VAE) framework.
  arXiv  Detail & Related papers  (2024-03-12T06:28:41Z)
• Distribution-Flexible Subset Quantization for Post-Quantizing Super-Resolution Networks [68.83451203841624]
  This paper introduces Distribution-Flexible Subset Quantization (DFSQ), a post-training quantization method for super-resolution networks. DFSQ conducts channel-wise normalization of the activations and applies distribution-flexible subset quantization (SQ) It achieves comparable performance to full-precision counterparts on 6- and 8-bit quantization, and incurs only a 0.1 dB PSNR drop on 4-bit quantization.
  arXiv  Detail & Related papers  (2023-05-10T04:19:11Z)
• Variational Gibbs State Preparation on NISQ devices [1.6600832946471173]
  We propose a variational quantum algorithm (VQA) to prepare Gibbs states of a quantum many-body system. The novelty of our VQA consists in implementing a parameterized quantum circuit acting on two distinct, yet connected, quantum registers. We benchmark our VQA by preparing Gibbs states of the transverse field Ising and Heisenberg XXZ models and achieve remarkably high fidelities.
  arXiv  Detail & Related papers  (2023-03-20T17:09:34Z)
• A kernel-based quantum random forest for improved classification [0.0]
  Quantum Machine Learning (QML) to enhance traditional classical learning methods has seen various limitations to its realisation. We extend the linear quantum support vector machine (QSVM) with kernel function computed through quantum kernel estimation (QKE) To limit overfitting, we further extend the model to employ a low-rank Nystr"om approximation to the kernel matrix.
  arXiv  Detail & Related papers  (2022-10-05T15:57:31Z)
• A Convergence Theory for Over-parameterized Variational Quantum Eigensolvers [21.72347971869391]
  The Variational Quantum Eigensolver (VQE) is a promising candidate for quantum applications on near-term Noisy Intermediate-Scale Quantum (NISQ) computers. We provide the first rigorous analysis of the convergence of VQEs in the over- parameterization regime.
  arXiv  Detail & Related papers  (2022-05-25T04:06:50Z)
• A Variational Quantum Attack for AES-like Symmetric Cryptography [69.80357450216633]
  We propose a variational quantum attack algorithm (VQAA) for classical AES-like symmetric cryptography. In the VQAA, the known ciphertext is encoded as the ground state of a Hamiltonian that is constructed through a regular graph.
  arXiv  Detail & Related papers  (2022-05-07T03:15:15Z)
• Q-Match: Iterative Shape Matching via Quantum Annealing [64.74942589569596]
  Finding shape correspondences can be formulated as an NP-hard quadratic assignment problem (QAP) This paper proposes Q-Match, a new iterative quantum method for QAPs inspired by the alpha-expansion algorithm. Q-Match can be applied for shape matching problems iteratively, on a subset of well-chosen correspondences, allowing us to scale to real-world problems.
  arXiv  Detail & Related papers  (2021-05-06T17:59:38Z)
• Efficient measure for the expressivity of variational quantum algorithms [72.59790225766777]
  We exploit an advanced tool in statistical learning theory, i.e., covering number, to study the expressivity of variational quantum algorithms. We first exhibit how the expressivity of VQAs with an arbitrary ansatze is upper bounded by the number of quantum gates and the measurement observable. We then explore the expressivity of VQAs on near-term quantum chips, where the system noise is considered.
  arXiv  Detail & Related papers  (2021-04-20T13:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.