DIFFCZSL: Compositional Zero-Shot Learning Regularized by Diffusion Representations

• URL: http://arxiv.org/abs/2608.19871v1
• Date: Thu, 20 Aug 2026 10:30:18 GMT
• Title: DIFFCZSL: Compositional Zero-Shot Learning Regularized by Diffusion Representations
• Abstract summary: Compositional Zero-Shot Learning (CZSL) aims to recognize unseen-object compositions by leveraging knowledge of primitive concepts learned from seen compositions.<n>We propose DIFFCZSL, a diffusion-augmented framework that injects generative priors from pre-trained diffusion models into CLIP-based CZSL pipelines.
• Score: 9.55658957185471
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compositional Zero-Shot Learning (CZSL) aims to recognize unseen attribute-object compositions by leveraging knowledge of primitive concepts learned from seen compositions. Although recent works achieve impressive performance in CZSL by leveraging large vision-language models, they primarily rely on discriminative representations that may not explicitly preserve the structured relationships between primitive concepts and their compositions. Motivated by the recent success of diffusion-based classifiers and their competitive performance relative to discriminative models, we investigate whether intermediate diffusion representations can provide complementary cues for CZSL. To this end, we propose DIFFCZSL, a diffusion-augmented framework that injects generative priors from pre-trained diffusion models into CLIP-based CZSL pipelines. We extract intermediate diffusion representations and project them into the CLIP embedding space to provide auxiliary supervision on both image and text modalities. Through contrastive alignment between CLIP embeddings and diffusion features during training, our method encourages the embedding geometry toward richer composition-aware semantics, while introducing no additional cost at inference time. Extensive experiments on three public CZSL benchmarks demonstrate consistent improvements over strong CLIP-based baselines under both closed-world and open-world settings. Our results highlight the complementary strengths of generative diffusion representations and discriminative vision-language models for compositional generalization.

Related papers

• Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers [65.70472330370735]
  We propose a new Language-Assisted Image Clustering (LAIC) framework with two complementary components.<n>We exploit cross-language relations to produce more discriminative self-supervision signals for clustering.<n>We learn category-wise continuous semantic centers via prompt learning to produce the final clustering assignments.
  arXiv  Detail & Related papers  (2026-03-25T13:08:14Z)
• Generalized Decoupled Learning for Enhancing Open-Vocabulary Dense Perception [71.26728044621458]
  DeCLIP is a novel framework that enhances CLIP by decoupling the self-attention module to obtain content'' and context'' features respectively.<n>It consistently achieves state-of-the-art performance across a broad spectrum of tasks, including 2D detection and segmentation, 3D instance segmentation, video instance segmentation, and 6D object pose estimation.
  arXiv  Detail & Related papers  (2025-08-15T06:43:51Z)
• Learning Clustering-based Prototypes for Compositional Zero-shot Learning [56.57299428499455]
  ClusPro is a robust clustering-based prototype mining framework for Compositional Zero-Shot Learning.<n>It defines the conceptual boundaries of primitives through a set of diversified prototypes.<n>ClusPro efficiently performs prototype clustering in a non-parametric fashion without the introduction of additional learnable parameters.
  arXiv  Detail & Related papers  (2025-02-10T14:20:01Z)
• Learning Visual Proxy for Compositional Zero-Shot Learning [18.38505448611429]
  We introduce Visual Proxy Learning, a method that reduces modality gaps and enhances compositional generalization.<n>We also propose Cross-Modal Joint Learning, which imposes cross-modal constraints between the text-image and fine-grained visual spaces.<n> Experiments show state-of-the-art performance in closed-world scenarios and competitive results in open-world settings.
  arXiv  Detail & Related papers  (2025-01-23T17:30:27Z)
• ResCLIP: Residual Attention for Training-free Dense Vision-language Inference [27.551367463011008]
  Cross-correlation of self-attention in CLIP's non-final layers also exhibits localization properties. We propose the Residual Cross-correlation Self-attention (RCS) module, which leverages the cross-correlation self-attention from intermediate layers to remold the attention in the final block. The RCS module effectively reorganizes spatial information, unleashing the localization potential within CLIP for dense vision-language inference.
  arXiv  Detail & Related papers  (2024-11-24T14:14:14Z)
• High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study [64.06777376676513]
  We develop a few-shot segmentation (FSS) framework based on foundation models. To be specific, we propose a simple approach to extract implicit knowledge from foundation models to construct coarse correspondence. Experiments on two widely used datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-10T08:04:11Z)
• ProxyCLIP: Proxy Attention Improves CLIP for Open-Vocabulary Segmentation [32.852004564832455]
  Open-vocabulary semantic segmentation requires models to integrate visual representations with semantic labels. This paper introduces ProxyCLIP, a framework designed to harmonize the strengths of Contrastive Language-Image Pre-training (CLIP) and Vision Foundation Models (VFMs) As a training-free approach, ProxyCLIP significantly improves the average mean Intersection over Union (mIoU) across eight benchmarks from 40.3 to 44.4.
  arXiv  Detail & Related papers  (2024-08-09T06:17:00Z)
• Prompting Language-Informed Distribution for Compositional Zero-Shot Learning [73.49852821602057]
  Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts. We propose a model by prompting the language-informed distribution, aka., PLID, for the task. Experimental results on MIT-States, UT-Zappos, and C-GQA datasets show the superior performance of the PLID to the prior arts.
  arXiv  Detail & Related papers  (2023-05-23T18:00:22Z)
• ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning [29.591615811894265]
  Open-World Compositional Zero-shot Learning (OW-CZSL) aims to recognize novel compositions of state and object primitives in images with no priors on the compositional space. We propose a novel method, termed Progressive Cross-primitive Compatibility (ProCC), to mimic the human learning process for OW-CZSL tasks.
  arXiv  Detail & Related papers  (2022-11-19T10:09:46Z)
• Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods [61.49061000562676]
  We introduce Cluster Learnability (CL) to assess learnability. CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means. We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
  arXiv  Detail & Related papers  (2022-06-02T19:05:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.