Compositional Concept Generalization with Variational Quantum Circuits

• URL: http://arxiv.org/abs/2509.09541v1
• Date: Thu, 11 Sep 2025 15:34:33 GMT
• Title: Compositional Concept Generalization with Variational Quantum Circuits
• Abstract summary: Previous work examined whether a compositional tensor-based sentence semantics can overcome the challenge, but led to negative results.<n>We conjecture that the increased training efficiency of quantum models will improve performance in these tasks.<n>We interpret the representations of compositional tensor-based models in Hilbert spaces and train Variational Quantum Circuits to learn these representations on an image captioning task.
• Score: 2.1533342461940608
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compositional generalization is a key facet of human cognition, but lacking in current AI tools such as vision-language models. Previous work examined whether a compositional tensor-based sentence semantics can overcome the challenge, but led to negative results. We conjecture that the increased training efficiency of quantum models will improve performance in these tasks. We interpret the representations of compositional tensor-based models in Hilbert spaces and train Variational Quantum Circuits to learn these representations on an image captioning task requiring compositional generalization. We used two image encoding techniques: a multi-hot encoding (MHE) on binary image vectors and an angle/amplitude encoding on image vectors taken from the vision-language model CLIP. We achieve good proof-of-concept results using noisy MHE encodings. Performance on CLIP image vectors was more mixed, but still outperformed classical compositional models.

Related papers

• Quantum Models with Multi-Stage Training for Compositional Concept Generalization [2.205782062842962]
  We provide a solution using a compositional model of meaning that separates nouns from relations and uses tensors and variational quantum circuits to train them on data.<n>For text, we work with vector representations of nouns and higher order tensor representations of relations using a set of different ansatz.<n>For images, we work with quantum encodings of image embeddings dervied from Open AI's Vision Language tool CLIP and contrast amplitude encoding, which preserves the original embedding geometry, with angle encoding, which introduces nonlinear feature transformations.
  arXiv  Detail & Related papers  (2026-08-16T07:58:46Z)
• Beyond Self-Attention: Sub-Quadratic Vision Transformers for Fast Image Captioning [2.33474141211381]
  The proposed model focuses on improving computational efficiency by restructuring the vision transformer architecture.<n>The model is evaluated on the Flickr 30K dataset, demonstrating competitive and significant improvement over existing works.
  arXiv  Detail & Related papers  (2026-06-07T16:57:24Z)
• Independent Density Estimation [27.51041148291178]
  We propose a new method called Independent Density Estimation (IDE) to tackle this challenge.<n>IDE aims to learn the connec- tion between individual words in a sentence and the corresponding features in an image, enabling compositional generalization.<n>Our models exhibit superior generalization to unseen compositions compared to current models when evaluated on various datasets.
  arXiv  Detail & Related papers  (2025-12-10T20:43:03Z)
• MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer [90.72238747690972]
  We present Manzano, a simple and scalable unified framework for multimodal large language models.<n>A single vision encoder feeds two adapters that produce continuous embeddings for image-to-text understanding and discrete tokens for text-to-image generation.<n>A unified autoregressive LLM predicts high-level semantics in the form of text and image tokens, with an auxiliary diffusion decoder subsequently translating the image tokens into pixels.
  arXiv  Detail & Related papers  (2025-09-19T17:58:00Z)
• SQ-GAN: Semantic Image Communications Using Masked Vector Quantization [54.35918290143049]
  This work introduces Semantically Masked Vector Quantized Generative Adversarial Network (SQ-GAN)<n>It is a novel approach integrating semantically driven image coding and vector quantization to optimize image compression for semantic/task-oriented communications.<n>SQ-GAN outperforms state-of-the-art image compression schemes such as JPEG2000, BPG, and deep-learning based methods across multiple metrics.
  arXiv  Detail & Related papers  (2025-02-13T17:35:57Z)
• GaussianToken: An Effective Image Tokenizer with 2D Gaussian Splatting [64.84383010238908]
  We propose an effective image tokenizer with 2D Gaussian Splatting as a solution.<n>In general, our framework integrates the local influence of 2D Gaussian distribution into the discrete space.<n> Competitive reconstruction performances on CIFAR, Mini-Net, and ImageNet-1K demonstrate the effectiveness of our framework.
  arXiv  Detail & Related papers  (2025-01-26T17:56:11Z)
• CAT: Content-Adaptive Image Tokenization [92.2116487267877]
  We introduce Content-Adaptive Tokenizer (CAT), which adjusts representation capacity based on the image content and encodes simpler images into fewer tokens.<n>We design a caption-based evaluation system that leverages large language models (LLMs) to predict content complexity and determine the optimal compression ratio for a given image.<n>By optimizing token allocation, CAT improves the FID score over fixed-ratio baselines trained with the same flops and boosts the inference throughput by 18.5%.
  arXiv  Detail & Related papers  (2025-01-06T16:28:47Z)
• Translatotron-V(ison): An End-to-End Model for In-Image Machine Translation [81.45400849638347]
  In-image machine translation (IIMT) aims to translate an image containing texts in source language into an image containing translations in target language. In this paper, we propose an end-to-end IIMT model consisting of four modules. Our model achieves competitive performance compared to cascaded models with only 70.9% of parameters, and significantly outperforms the pixel-level end-to-end IIMT model.
  arXiv  Detail & Related papers  (2024-07-03T08:15:39Z)
• Kosmos-G: Generating Images in Context with Multimodal Large Language Models [117.0259361818715]
  Current subject-driven image generation methods require test-time tuning and cannot accept interleaved multi-image and text input. This paper presents Kosmos-G, a model that leverages the advanced multimodal perception capabilities of Multimodal Large Language Models. Kosmos-G demonstrates an impressive capability of zero-shot subject-driven generation with interleaved multi-image and text input.
  arXiv  Detail & Related papers  (2023-10-04T17:28:44Z)
• Image Compression with Product Quantized Masked Image Modeling [44.15706119017024]
  Recent neural compression methods have been based on the popular hyperprior framework. It relies on Scalar Quantization and offers a very strong compression performance. This contrasts from recent advances in image generation and representation learning, where Vector Quantization is more commonly employed.
  arXiv  Detail & Related papers  (2022-12-14T17:50:39Z)
• Prompt-based Learning for Unpaired Image Captioning [86.44188293709307]
  Unpaired Image Captioning (UIC) has been developed to learn image descriptions from unaligned vision-language sample pairs. Recent successes of Vision-Language Pre-Trained Models (VL-PTMs) have triggered the development of prompt-based learning. We present in this paper a novel scheme based on prompt to train the UIC model, making best use of the powerful generalization ability.
  arXiv  Detail & Related papers  (2022-05-26T03:13:43Z)
• Variational Quanvolutional Neural Networks with enhanced image encoding [0.0]
  We study the effect of three different quantum image encoding approaches on the performance of a convolution-inspired hybrid quantum-classical image classification algorithm called quanvolutional neural network (QNN) Our experiments indicate that some image encodings are better suited for variational circuits.
  arXiv  Detail & Related papers  (2021-06-14T12:08:30Z)
• A Flexible Framework for Designing Trainable Priors with Adaptive Smoothing and Game Encoding [57.1077544780653]
  We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems. We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions. This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
  arXiv  Detail & Related papers  (2020-06-26T08:34:54Z)
• Non-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning [46.060954649681385]
  We propose a Non-Autoregressive Image Captioning model with a novel training paradigm: Counterfactuals-critical Multi-Agent Learning (CMAL) Our NAIC model achieves a performance comparable to state-of-the-art autoregressive models, while brings 13.9x decoding speedup.
  arXiv  Detail & Related papers  (2020-05-10T15:09:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.