COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

• URL: http://arxiv.org/abs/2606.28696v1
• Date: Sat, 27 Jun 2026 02:43:13 GMT
• Title: COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models
• Abstract summary: We present the first unified multimodal framework that grounds composition-intent control in a single system spanning both composition perception and composition-guided generation.<n>We construct Comp-11, a large-scale dataset with an 11-class taxonomy and reasoning-augmented annotations.
• Score: 52.024920838442576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composition is a high-level visual intent that governs where subjects are placed and how a scene is organized, yet current unified multimodal models remain unreliable at fine-grained composition recognition and struggle to turn such intent into controllable generation. We present COMPASS, the first unified multimodal framework that grounds composition-intent control in a single system spanning both composition perception and composition-guided generation, with a shared expert token $τ_c$ as the central intent anchor. On the perception side, COMPASS injects composition expertise into an MoE backbone in a minimally invasive manner and distills the inferred intent into $τ_c$. On the generation side, COMPASS reuses $τ_c$ as a global conditioning signal that steers the denoising trajectory, effectively converting passive composition analysis into explicit layout control. To support systematic instruction-following composition learning and evaluation at scale, we construct Comp-11, a large-scale dataset with an 11-class taxonomy and reasoning-augmented annotations. Extensive experiments show that COMPASS substantially improves category-level composition understanding and delivers more composition-consistent, prompt-faithful generation than strong baselines.

Related papers

• AC-VLA: Robust Out-of-Distribution Action Execution via Compositional Learning [52.47988016454267]
  Vision-Language-Action (VLA) models excel at end-to-end robotic manipulation but struggle with out-of-distribution (OOD) generalization.<n>We identify two mutually reinforcing modes: emphtrajectory overfitting, where models overfit to holistic trajectory patterns rather than compositional sub-skill semantics; and emphperceptual shortcut, where action tokens over-rely on wrist-view textures at the expense of global spatial grounding.<n>We introduce textbfAC-VLA, a plug-and-play Action Compositional learning
  arXiv  Detail & Related papers  (2026-07-17T07:51:03Z)
• Hierarchical Contrastive Learning for Multimodal Data [3.1788482285024107]
  Hierarchical Contrastive Learning is a framework that learns globally shared, partially shared, and modality-specific representations.<n>On multimodal electronic health records, HCL yields more informative representations and consistently improves predictive performance.
  arXiv  Detail & Related papers  (2026-04-07T05:53:04Z)
• Bilateral Intent-Enhanced Sequential Recommendation with Embedding Perturbation-Based Contrastive Learning [1.7522809149611367]
  Bilateral Intent-enhanced Embedding Perturbation-based Contrastive Learning framework.<n>BIPCL integrates multi-intent signals into both item and sequence representations via a bilateral intent-enhancement mechanism.<n> experiments on benchmark datasets demonstrate that BIPCL consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-04-03T07:55:04Z)
• PartComposer: Learning and Composing Part-Level Concepts from Single-Image Examples [31.17687322390452]
  PartComposer is a framework for part-level concept learning from single-image examples.<n>We propose a dynamic data synthesis pipeline generating diverse part compositions to address one-shot data scarcity.
  arXiv  Detail & Related papers  (2025-06-03T15:43:28Z)
• Consistency of Compositional Generalization across Multiple Levels [31.77432446850103]
  We propose a meta-learning based framework, for achieving consistent compositional generalization across multiple levels.<n>We build a GQA-CCG dataset to quantitatively evaluate the consistency.
  arXiv  Detail & Related papers  (2024-12-18T09:09:41Z)
• Enhancing Graph Contrastive Learning with Reliable and Informative Augmentation for Recommendation [84.45144851024257]
  We propose a novel framework that aims to enhance graph contrastive learning by constructing contrastive views with stronger collaborative information via discrete codes.<n>The core idea is to map users and items into discrete codes rich in collaborative information for reliable and informative contrastive view generation.
  arXiv  Detail & Related papers  (2024-09-09T14:04:17Z)
• IntCoOp: Interpretability-Aware Vision-Language Prompt Tuning [94.52149969720712]
  IntCoOp learns to jointly align attribute-level inductive biases and class embeddings during prompt-tuning. IntCoOp improves CoOp by 7.35% in average performance across 10 diverse datasets.
  arXiv  Detail & Related papers  (2024-06-19T16:37:31Z)
• CSCNET: Class-Specified Cascaded Network for Compositional Zero-Shot Learning [62.090051975043544]
  Attribute and object (A-O) disentanglement is a fundamental and critical problem for Compositional Zero-shot Learning (CZSL) We propose a novel A-O disentangled framework for CZSL, namely Class-specified Cascaded Network (CSCNet)
  arXiv  Detail & Related papers  (2024-03-09T14:18:41Z)
• Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representations [70.41385310930846]
  We present an end-to-end framework Structure-CLIP to enhance multi-modal structured representations. We use scene graphs to guide the construction of semantic negative examples, which results in an increased emphasis on learning structured representations. A Knowledge-Enhance (KEE) is proposed to leverage SGK as input to further enhance structured representations.
  arXiv  Detail & Related papers  (2023-05-06T03:57:05Z)
• HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction [24.853265244512954]
  We propose a hierarchical contrastive learning Framework for DistantlySupervised relation extraction (HiCLRE) to reduce noisy sentences. Specifically, we propose a three-level hierarchical learning framework to interact with cross levels, generating the de-noising context-aware representations. Experiments demonstrate that HiCLRE significantly outperforms strong baselines in various mainstream DSRE datasets.
  arXiv  Detail & Related papers  (2022-02-27T12:48:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.