Mutual Balancing in State-Object Components for Compositional Zero-Shot Learning

• URL: http://arxiv.org/abs/2211.10647v1
• Date: Sat, 19 Nov 2022 10:21:22 GMT
• Title: Mutual Balancing in State-Object Components for Compositional Zero-Shot Learning
• Authors: Chenyi Jiang (1), Dubing Chen (1), Shidong Wang (2), Yuming Shen (3), Haofeng Zhang (1), Ling Shao (4) ((1) Nanjing University of Science and Technology, (2) University of Newcastle-upon-Tyne, (3) University of Oxford, (4) Terminus Group, Beijing, China)
• Abstract summary: Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions from seen states and objects. We propose a novel method called MUtual balancing in STate-object components (MUST) for CZSL, which provides a balancing inductive bias for the model. Our approach significantly outperforms the state-of-the-art on MIT-States, UT-Zappos, and C-GQA when combined with the basic CZSL frameworks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions from seen states and objects. The disparity between the manually labeled semantic information and its actual visual features causes a significant imbalance of visual deviation in the distribution of various object classes and state classes, which is ignored by existing methods. To ameliorate these issues, we consider the CZSL task as an unbalanced multi-label classification task and propose a novel method called MUtual balancing in STate-object components (MUST) for CZSL, which provides a balancing inductive bias for the model. In particular, we split the classification of the composition classes into two consecutive processes to analyze the entanglement of the two components to get additional knowledge in advance, which reflects the degree of visual deviation between the two components. We use the knowledge gained to modify the model's training process in order to generate more distinct class borders for classes with significant visual deviations. Extensive experiments demonstrate that our approach significantly outperforms the state-of-the-art on MIT-States, UT-Zappos, and C-GQA when combined with the basic CZSL frameworks, and it can improve various CZSL frameworks. Our codes are available on https://anonymous.4open.science/r/MUST_CGE/.

Related papers

• Cross-composition Feature Disentanglement for Compositional Zero-shot Learning [49.919635694894204]
  Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL) We propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs and constrains the disentangled primitive features to be general across these compositions.
  arXiv  Detail & Related papers  (2024-08-19T08:23:09Z)
• Revealing the Proximate Long-Tail Distribution in Compositional Zero-Shot Learning [20.837664254230567]
  Compositional Zero-Shot Learning (CZSL) aims to transfer knowledge from seen state-object pairs to novel pairs. Visual bias caused by predictions of combinations of state-objects blurs their visual features hindering learning of distinguishable class prototypes. We mathematically deduce the role of class prior within the long-tailed distribution in CZSL. Building upon this insight, we incorporate visual bias caused by compositions into the classifier's training and inference by estimating it as a proximate class prior.
  arXiv  Detail & Related papers  (2023-12-26T07:35:02Z)
• Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning [52.506434446439776]
  Compositional zero-shot learning (CZSL) aims to recognize compositions with prior knowledge of known primitives (attribute and object) We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL.
  arXiv  Detail & Related papers  (2023-08-08T03:24:21Z)
• Prompting Language-Informed Distribution for Compositional Zero-Shot Learning [73.49852821602057]
  Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts. We propose a model by prompting the language-informed distribution, aka., PLID, for the task. Experimental results on MIT-States, UT-Zappos, and C-GQA datasets show the superior performance of the PLID to the prior arts.
  arXiv  Detail & Related papers  (2023-05-23T18:00:22Z)
• Siamese Contrastive Embedding Network for Compositional Zero-Shot Learning [76.13542095170911]
  Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions formed from seen state and object during training. We propose a novel Siamese Contrastive Embedding Network (SCEN) for unseen composition recognition. Our method significantly outperforms the state-of-the-art approaches on three challenging benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-29T09:02:35Z)
• Disentangled Ontology Embedding for Zero-shot Learning [39.014714187825646]
  Knowledge Graph (KG) and its variant of ontology have been widely used for knowledge representation, and have shown to be quite effective in augmenting Zero-shot Learning (ZSL) Existing ZSL methods that utilize KGs all neglect the complexity of inter-class relationships represented in KGs. In this paper, we focus on for augmenting ZSL, and propose to learn disentangled ontology embeddings guided by semantic properties. We also contribute a new ZSL framework named DOZSL, which contains two new ZSL solutions based on generative models and graph propagation models.
  arXiv  Detail & Related papers  (2022-06-08T08:29:30Z)
• KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning [52.422873819371276]
  The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images. Here, we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. We estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL.
  arXiv  Detail & Related papers  (2022-05-13T17:18:15Z)
• OntoZSL: Ontology-enhanced Zero-shot Learning [19.87808305218359]
  Key to implementing Zero-shot Learning (ZSL) is to leverage the prior knowledge of classes which builds the semantic relationship between classes. In this paper, we explore richer and more competitive prior knowledge to model the inter-class relationship for ZSL. To address the data imbalance between seen classes and unseen classes, we developed a generative ZSL framework with Generative Adversarial Networks (GANs)
  arXiv  Detail & Related papers  (2021-02-15T04:39:58Z)
• Information Bottleneck Constrained Latent Bidirectional Embedding for Zero-Shot Learning [59.58381904522967]
  We propose a novel embedding based generative model with a tight visual-semantic coupling constraint. We learn a unified latent space that calibrates the embedded parametric distributions of both visual and semantic spaces. Our method can be easily extended to transductive ZSL setting by generating labels for unseen images.
  arXiv  Detail & Related papers  (2020-09-16T03:54:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.