KamonBench: A Grammar-Based Dataset for Evaluating Compositional Factor Recovery in Vision-Language Models

• URL: http://arxiv.org/abs/2605.13322v2
• Date: Sun, 17 May 2026 01:47:06 GMT
• Title: KamonBench: A Grammar-Based Dataset for Evaluating Compositional Factor Recovery in Vision-Language Models
• Abstract summary: Kamon (family crests) are an important part of Japanese culture and a natural test case for compositional visual recognition.<n>We introduce KamonBench, a grammar-based image-to-structure benchmark with 20,000 synthetic composite crests and auxiliary component examples.
• Score: 5.049057348282933
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Kamon (family crests) are an important part of Japanese culture and a natural test case for compositional visual recognition: each crest combines a small number of symbolic choices, but the space of possible descriptions is sparse. We introduce KamonBench, a grammar-based image-to-structure benchmark with 20,000 synthetic composite crests and auxiliary component examples. Each composite crest is paired with a formal kamon description language - "kamon yōgo" - description, a segmented Japanese analysis, an English translation, and a non-linguistic program code. Because each synthetic crest is generated from known factors, namely container, modifier, and motif, KamonBench supports evaluation beyond caption-level accuracy: direct program-code factor metrics, controlled factor-pair recombination splits, counterfactual motif-sensitivity groups under fixed container-modifier contexts, and linear probes of factor accessibility. We include baseline results for a ViT encoder/Transformer decoder and two VGG n-gram decoders, with and without learned positional masks. KamonBench therefore provides a controlled testbed for sparse compositional visual recognition and factor recovery in vision-language models.

Related papers

• Full Glyph Images Beat Token Embeddings: A Controlled Study for Transformers [1.920312875157821]
  We use a Vision-based model and an index-based baseline to represent Chinese characters.<n>Across all tested decoder backbones, the Vision-based model consistently outperforms the baseline.<n>The advantage requires both spatially coherent input and a ViT encoder with 2D positional encodings.
  arXiv  Detail & Related papers  (2026-07-04T19:30:16Z)
• Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation [16.594496639787195]
  Open-vocabulary segmentation models often struggle to generalize to unseen combinations of categories object and attributes.<n>We propose a Decomposed Vision-Language Alignment framework that factorizes textual prompts into a concept token and multiple attribute tokens.<n>At the feature level, we introduce a Feature-Gated Cross-Attention module that generates attribute-specific gating maps to fuse information in a multiplicative manner.
  arXiv  Detail & Related papers  (2026-05-15T13:27:01Z)
• Communication-Inspired Tokenization for Structured Image Representations [74.17163003465537]
  COMmunication inspired Tokenization (COMiT) is a framework for learning structured discrete visual token sequences.<n>Our experiments demonstrate that while semantic alignment provides grounding, attentive sequential tokenization is critical for inducing interpretable, object-centric token structure.
  arXiv  Detail & Related papers  (2026-02-24T09:53:50Z)
• In-Context Compositional Learning via Sparse Coding Transformer [21.27564345380246]
  We propose a reformulation of the attention to enhance its capability for compositional tasks.<n>In sparse coding, data are represented as sparse combinations of dictionary atoms with coefficients that capture their compositional rules.<n>We demonstrate the effectiveness of our approach on the S-RAVEN and RAVEN datasets.
  arXiv  Detail & Related papers  (2025-11-25T11:19:58Z)
• Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment [8.097278579432908]
  The choice of tokenizer algorithm is the most significant factor influencing performance, with Unigram-based tokenizers consistently outperforming BPE across most settings.<n>While better morphological alignment shows a moderate, positive correlation with performance on text classification and structure prediction tasks, its impact is secondary to the tokenizer algorithm.
  arXiv  Detail & Related papers  (2025-08-11T19:23:59Z)
• Multigranular Evaluation for Brain Visual Decoding [5.19485079754946]
  Existing evaluation protocols for brain visual decoding rely on coarse metrics that obscure inter-model differences, lack neuroscientific foundation, and fail to capture fine-grained visual distinctions.<n>We introduce BASIC, a unified, multigranular evaluation framework that jointly quantifies structural fidelity, inferential alignment, and contextual coherence between decoded and ground truth images.<n>For the structural level, we introduce a hierarchical suite of segmentation-based metrics, including foreground, semantic, instance, and component masks, anchored in granularity-aware correspondence across mask structures.<n>For the semantic level, we extract structured scene representations encompassing objects, attributes, and relationships using multimodal large
  arXiv  Detail & Related papers  (2025-07-10T17:59:24Z)
• CoLa: Chinese Character Decomposition with Compositional Latent Components [63.01848135436417]
  Humans can decompose Chinese characters into compositional components and recombine them to recognize unseen characters.<n>We propose a deep latent variable model that learns Compositional Latent components of Chinese characters (CoLa) without relying on human-defined decomposition schemes.
  arXiv  Detail & Related papers  (2025-06-04T10:06:15Z)
• Graph-guided Cross-composition Feature Disentanglement for Compositional Zero-shot Learning [54.08741382593959]
  Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL)<n>It is challenging to learn disentangled primitive features that are general across different compositions.<n>We propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs.
  arXiv  Detail & Related papers  (2024-08-19T08:23:09Z)
• Text encoders bottleneck compositionality in contrastive vision-language models [76.2406963762722]
  We train text-only recovery probes that aim to reconstruct captions from single-vector text representations. We find that CLIP's text encoder falls short on more compositional inputs. Results suggest text-only recoverability is a necessary (but not sufficient) condition for modeling compositional factors.
  arXiv  Detail & Related papers  (2023-05-24T08:48:44Z)
• Compositional Generalization in Grounded Language Learning via Induced Model Sparsity [81.38804205212425]
  We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations. We design an agent that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal. Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations.
  arXiv  Detail & Related papers  (2022-07-06T08:46:27Z)
• Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks [86.10875837475783]
  Systematic compositionality is an essential mechanism in human language, allowing the recombination of known parts to create novel expressions. Existing neural models have been shown to lack this basic ability in learning symbolic structures. We propose two auxiliary sequence prediction tasks that track the progress of function and argument semantics.
  arXiv  Detail & Related papers  (2021-09-30T16:41:19Z)
• Image Captioning with Compositional Neural Module Networks [18.27510863075184]
  We introduce a hierarchical framework for image captioning that explores both compositionality and sequentiality of natural language. Our algorithm learns to compose a detail-rich sentence by selectively attending to different modules corresponding to unique aspects of each object detected in an input image.
  arXiv  Detail & Related papers  (2020-07-10T20:58:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.