Learning Graph Embeddings for Open World Compositional Zero-Shot Learning

• URL: http://arxiv.org/abs/2105.01017v1
• Date: Mon, 3 May 2021 17:08:21 GMT
• Title: Learning Graph Embeddings for Open World Compositional Zero-Shot Learning
• Authors: Massimiliano Mancini, Muhammad Ferjad Naeem, Yongqin Xian, Zeynep Akata
• Abstract summary: Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. We propose a new approach, Compositional Cosine Graph Embeddings (Co-CGE) Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network.
• Score: 47.09665742252187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compositional Zero-Shot learning (CZSL) aims to recognize unseen compositions of state and object visual primitives seen during training. A problem with standard CZSL is the assumption of knowing which unseen compositions will be available at test time. In this work, we overcome this assumption operating on the open world setting, where no limit is imposed on the compositional space at test time, and the search space contains a large number of unseen compositions. To address this problem, we propose a new approach, Compositional Cosine Graph Embeddings (Co-CGE), based on two principles. First, Co-CGE models the dependency between states, objects and their compositions through a graph convolutional neural network. The graph propagates information from seen to unseen concepts, improving their representations. Second, since not all unseen compositions are equally feasible, and less feasible ones may damage the learned representations, Co-CGE estimates a feasibility score for each unseen composition, using the scores as margins in a cosine similarity-based loss and as weights in the adjacency matrix of the graphs. Experiments show that our approach achieves state-of-the-art performances in standard CZSL while outperforming previous methods in the open world scenario.

Related papers

• Cross-composition Feature Disentanglement for Compositional Zero-shot Learning [49.919635694894204]
  Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL) We propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs and constrains the disentangled primitive features to be general across these compositions.
  arXiv  Detail & Related papers  (2024-08-19T08:23:09Z)
• Simple Primitives with Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-shot Learning [86.5258816031722]
  The task of Compositional Zero-Shot Learning (CZSL) is to recognize images of novel state-object compositions that are absent during the training stage. Previous methods of learning compositional embedding have shown effectiveness in closed-world CZSL. In Open-World CZSL (OW-CZSL), their performance tends to degrade significantly due to the large cardinality of possible compositions.
  arXiv  Detail & Related papers  (2022-11-05T12:57:06Z)
• Reference-Limited Compositional Zero-Shot Learning [19.10692212692771]
  Compositional zero-shot learning (CZSL) refers to recognizing unseen compositions of known visual primitives. We propose a novel Meta Compositional Graph Learner (MetaCGL) that can efficiently learn the compositionality from insufficient referential information.
  arXiv  Detail & Related papers  (2022-08-22T03:58:02Z)
• KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning [52.422873819371276]
  The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images. Here, we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. We estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL.
  arXiv  Detail & Related papers  (2022-05-13T17:18:15Z)
• On Leveraging Variational Graph Embeddings for Open World Compositional Zero-Shot Learning [3.9348884623092517]
  We learn composition of primitive concepts, i.e. objects and states, in such a way that even their novel compositions can be zero-shot classified. We propose a Compositional Variational Graph Autoencoder (CVGAE) approach for learning the variational embeddings of the primitive concepts.
  arXiv  Detail & Related papers  (2022-04-23T13:30:08Z)
• Learning Graph Embeddings for Compositional Zero-shot Learning [73.80007492964951]
  In compositional zero-shot learning, the goal is to recognize unseen compositions of observed visual primitives states. We propose a novel graph formulation called Compositional Graph Embedding (CGE) that learns image features and latent representations of visual primitives in an end-to-end manner. By learning a joint compatibility that encodes semantics between concepts, our model allows for generalization to unseen compositions without relying on an external knowledge base like WordNet.
  arXiv  Detail & Related papers  (2021-02-03T10:11:03Z)
• Open World Compositional Zero-Shot Learning [47.09665742252187]
  Compositional Zero-Shot learning (CZSL) requires to recognize state-object compositions unseen during training. We operate on the open world setting, where the search space includes a large number of unseen compositions.
  arXiv  Detail & Related papers  (2021-01-29T14:45:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.