KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning

• URL: http://arxiv.org/abs/2205.06784v1
• Date: Fri, 13 May 2022 17:18:15 GMT
• Title: KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning
• Authors: Shyamgopal Karthik, Massimiliano Mancini, Zeynep Akata
• Abstract summary: The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images. Here, we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. We estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL.
• Score: 52.422873819371276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images, given only a subset of them during training and no prior on the unseen compositions. In this setting, models operate on a huge output space, containing all possible state-object compositions. While previous works tackle the problem by learning embeddings for the compositions jointly, here we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. To ensure that the model develops primitive-specific features, we equip the state and object classifiers with separate, non-linear feature extractors. Moreover, we estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Finally, we propose a new setting, i.e. CZSL under partial supervision (pCZSL), where either only objects or state labels are available during training, and we can use our prior to estimate the missing labels. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL, surpassing most recent competitors even when coupled with semi-supervised learning techniques. Code available at: https://github.com/ExplainableML/KG-SP.

Related papers

• Cross-composition Feature Disentanglement for Compositional Zero-shot Learning [49.919635694894204]
  Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL) We propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs and constrains the disentangled primitive features to be general across these compositions.
  arXiv  Detail & Related papers  (2024-08-19T08:23:09Z)
• Attention Based Simple Primitives for Open World Compositional Zero-Shot Learning [12.558701595138928]
  Compositional Zero-Shot Learning (CZSL) aims to predict unknown compositions made up of attribute and object pairs. We are exploring Open World Compositional Zero-Shot Learning (OW-CZSL) in this study, where our test space encompasses all potential combinations of attributes and objects. Our approach involves utilizing the self-attention mechanism between attributes and objects to achieve better generalization from seen to unseen compositions.
  arXiv  Detail & Related papers  (2024-07-18T17:11:29Z)
• HOMOE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts [25.930021907054797]
  We propose a novel framework that combines the Modern Hopfield Network with a Mixture of Experts to classify the compositions of previously unseen objects. Our approach achieves SOTA performance on several benchmarks, including MIT-States and UT-Zappos.
  arXiv  Detail & Related papers  (2023-11-23T07:32:20Z)
• Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning [52.506434446439776]
  Compositional zero-shot learning (CZSL) aims to recognize compositions with prior knowledge of known primitives (attribute and object) We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL.
  arXiv  Detail & Related papers  (2023-08-08T03:24:21Z)
• Prompting Language-Informed Distribution for Compositional Zero-Shot Learning [73.49852821602057]
  Compositional zero-shot learning (CZSL) task aims to recognize unseen compositional visual concepts. We propose a model by prompting the language-informed distribution, aka., PLID, for the task. Experimental results on MIT-States, UT-Zappos, and C-GQA datasets show the superior performance of the PLID to the prior arts.
  arXiv  Detail & Related papers  (2023-05-23T18:00:22Z)
• ProCC: Progressive Cross-primitive Compatibility for Open-World Compositional Zero-Shot Learning [29.591615811894265]
  Open-World Compositional Zero-shot Learning (OW-CZSL) aims to recognize novel compositions of state and object primitives in images with no priors on the compositional space. We propose a novel method, termed Progressive Cross-primitive Compatibility (ProCC), to mimic the human learning process for OW-CZSL tasks.
  arXiv  Detail & Related papers  (2022-11-19T10:09:46Z)
• Learning Graph Embeddings for Open World Compositional Zero-Shot Learning [47.09665742252187]
  Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. We propose a new approach, Compositional Cosine Graph Embeddings (Co-CGE) Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network.
  arXiv  Detail & Related papers  (2021-05-03T17:08:21Z)
• Open World Compositional Zero-Shot Learning [47.09665742252187]
  Compositional Zero-Shot learning (CZSL) requires to recognize state-object compositions unseen during training. We operate on the open world setting, where the search space includes a large number of unseen compositions.
  arXiv  Detail & Related papers  (2021-01-29T14:45:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.