WARM-CAT: Warm-Started Test-Time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning

• URL: http://arxiv.org/abs/2602.23114v2
• Date: Fri, 27 Feb 2026 07:09:26 GMT
• Title: WARM-CAT: Warm-Started Test-Time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning
• Authors: Xudong Yan, Songhe Feng, Jiaxin Wang, Xin Su, Yi Jin,
• Abstract summary: Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones.<n>We propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities to update multimodal prototypes at test time.<n>Our approach achieves state-of-the-art performance on four benchmark datasets under both closed-world and open-world settings.
• Score: 41.10398503450224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compositional Zero-Shot Learning (CZSL) aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones. Existing methods suffer from performance degradation caused by the distribution shift of label space at test time, which stems from the inclusion of unseen compositions recombined from attributes and objects. To overcome the challenge, we propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities from unsupervised data to update multimodal prototypes at test time. Building on this, we further design an adaptive update weight to control the degree of prototype adjustment, enabling the model to flexibly adapt to distribution shift during testing. Moreover, a dynamic priority queue is introduced that stores high-confidence images to acquire visual prototypes from historical images for inference. Since the model tends to favor compositions already stored in the queue during testing, we warm-start the queue by initializing it with training images for visual prototypes of seen compositions and generating unseen visual prototypes using the mapping learned between seen and unseen textual prototypes. Considering the semantic consistency of multimodal knowledge, we align textual and visual prototypes by multimodal collaborative representation learning. To provide a more reliable evaluation for CZSL, we introduce a new benchmark dataset, C-Fashion, and refine the widely used but noisy MIT-States dataset. Extensive experiments indicate that our approach achieves state-of-the-art performance on four benchmark datasets under both closed-world and open-world settings. The source code and datasets are available at https://github.com/xud-yan/WARM-CAT .

Related papers

• TOMCAT: Test-time Comprehensive Knowledge Accumulation for Compositional Zero-Shot Learning [35.14123452166428]
  Compositional Zero-Shot Learning aims to recognize novel attribute-object compositions based on the knowledge learned from seen ones.<n>Existing methods suffer from performance degradation caused by the distribution shift of label space at test time.<n>We propose a novel approach that accumulates comprehensive knowledge in both textual and visual modalities to update multimodal prototypes at test time.
  arXiv  Detail & Related papers  (2025-10-23T03:20:29Z)
• Dynamic Multimodal Prototype Learning in Vision-Language Models [44.84161970425967]
  We introduce textbfProtoMM, a training-free framework that constructs multimodal prototypes to adapt vision-language models during the test time.<n>By viewing the prototype as a discrete distribution over the textual descriptions and visual particles, ProtoMM has the ability to combine the multimodal features for comprehensive prototype learning.
  arXiv  Detail & Related papers  (2025-07-04T15:31:47Z)
• Cross-Modal Prototype Allocation: Unsupervised Slide Representation Learning via Patch-Text Contrast in Computational Pathology [10.811667603360041]
  ProAlign is a cross-modal unsupervised slide representation learning framework.<n>We leverage a large language model (LLM) to generate descriptive text for the prototype types present in a whole slide image.<n>We propose a parameter-free attention aggregation strategy that utilizes the similarity between patches and these prototypes to form unsupervised slide embeddings.
  arXiv  Detail & Related papers  (2025-03-26T03:31:07Z)
• Learning Visual Proxy for Compositional Zero-Shot Learning [18.38505448611429]
  We introduce Visual Proxy Learning, a method that reduces modality gaps and enhances compositional generalization.<n>We also propose Cross-Modal Joint Learning, which imposes cross-modal constraints between the text-image and fine-grained visual spaces.<n> Experiments show state-of-the-art performance in closed-world scenarios and competitive results in open-world settings.
  arXiv  Detail & Related papers  (2025-01-23T17:30:27Z)
• Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning [50.26965628047682]
  Adapting pre-trained models to open classes is a challenging problem in machine learning. In this paper, we consider combining the advantages of both and come up with a test-time prompt tuning approach. Our proposed method outperforms all comparison methods on average considering both base and new classes.
  arXiv  Detail & Related papers  (2024-08-29T12:34:01Z)
• In-context Prompt Learning for Test-time Vision Recognition with Frozen Vision-language Model [13.983810804606264]
  We propose In-Context Prompt Learning (InCPL) for test-time visual recognition tasks. InCPL associates a new test sample with very few labeled examples as context information. We introduce a context-aware unsupervised loss to optimize visual prompts tailored to test samples.
  arXiv  Detail & Related papers  (2024-03-10T08:15:51Z)
• With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning [47.96387857237473]
  We devise a network which can perform attention over activations obtained while processing other training samples. Our memory models the distribution of past keys and values through the definition of prototype vectors. We demonstrate that our proposal can increase the performance of an encoder-decoder Transformer by 3.7 CIDEr points both when training in cross-entropy only and when fine-tuning with self-critical sequence training.
  arXiv  Detail & Related papers  (2023-08-23T18:53:00Z)
• Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning [52.506434446439776]
  Compositional zero-shot learning (CZSL) aims to recognize compositions with prior knowledge of known primitives (attribute and object) We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL.
  arXiv  Detail & Related papers  (2023-08-08T03:24:21Z)
• Multi-View Class Incremental Learning [57.14644913531313]
  Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. This paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views.
  arXiv  Detail & Related papers  (2023-06-16T08:13:41Z)
• Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning [58.2091760793799]
  We propose a novel contrastive prototype learning with augmented embeddings (CPLAE) model. With a class prototype as an anchor, CPL aims to pull the query samples of the same class closer and those of different classes further away. Extensive experiments on several benchmarks demonstrate that our proposed CPLAE achieves new state-of-the-art.
  arXiv  Detail & Related papers  (2021-01-23T13:22:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.