HOMOE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts

• URL: http://arxiv.org/abs/2311.14747v1
• Date: Thu, 23 Nov 2023 07:32:20 GMT
• Title: HOMOE: A Memory-Based and Composition-Aware Framework for Zero-Shot Learning with Hopfield Network and Soft Mixture of Experts
• Authors: Do Huu Dat, Po Yuan Mao, Tien Hoang Nguyen, Wray Buntine, Mohammed Bennamoun
• Abstract summary: We propose a novel framework that combines the Modern Hopfield Network with a Mixture of Experts to classify the compositions of previously unseen objects. Our approach achieves SOTA performance on several benchmarks, including MIT-States and UT-Zappos.
• Score: 25.930021907054797
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compositional Zero-Shot Learning (CZSL) has emerged as an essential paradigm in machine learning, aiming to overcome the constraints of traditional zero-shot learning by incorporating compositional thinking into its methodology. Conventional zero-shot learning has difficulty managing unfamiliar combinations of seen and unseen classes because it depends on pre-defined class embeddings. In contrast, Compositional Zero-Shot Learning uses the inherent hierarchies and structural connections among classes, creating new class representations by combining attributes, components, or other semantic elements. In our paper, we propose a novel framework that for the first time combines the Modern Hopfield Network with a Mixture of Experts (HOMOE) to classify the compositions of previously unseen objects. Specifically, the Modern Hopfield Network creates a memory that stores label prototypes and identifies relevant labels for a given input image. Following this, the Mixture of Expert models integrates the image with the fitting prototype to produce the final composition classification. Our approach achieves SOTA performance on several benchmarks, including MIT-States and UT-Zappos. We also examine how each component contributes to improved generalization.

Related papers

• CSCNET: Class-Specified Cascaded Network for Compositional Zero-Shot Learning [62.090051975043544]
  Attribute and object (A-O) disentanglement is a fundamental and critical problem for Compositional Zero-shot Learning (CZSL) We propose a novel A-O disentangled framework for CZSL, namely Class-specified Cascaded Network (CSCNet)
  arXiv  Detail & Related papers  (2024-03-09T14:18:41Z)
• Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning [52.506434446439776]
  Compositional zero-shot learning (CZSL) aims to recognize compositions with prior knowledge of known primitives (attribute and object) We propose a simple and scalable framework called Composition Transformer (CoT) to address these issues. Our method achieves SoTA performance on several benchmarks, including MIT-States, C-GQA, and VAW-CZSL.
  arXiv  Detail & Related papers  (2023-08-08T03:24:21Z)
• Mutual Balancing in State-Object Components for Compositional Zero-Shot Learning [0.0]
  Compositional Zero-Shot Learning (CZSL) aims to recognize unseen compositions from seen states and objects. We propose a novel method called MUtual balancing in STate-object components (MUST) for CZSL, which provides a balancing inductive bias for the model. Our approach significantly outperforms the state-of-the-art on MIT-States, UT-Zappos, and C-GQA when combined with the basic CZSL frameworks.
  arXiv  Detail & Related papers  (2022-11-19T10:21:22Z)
• KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning [52.422873819371276]
  The goal of open-world compositional zero-shot learning (OW-CZSL) is to recognize compositions of state and objects in images. Here, we revisit a simple CZSL baseline and predict the primitives, i.e. states and objects, independently. We estimate the feasibility of each composition through external knowledge, using this prior to remove unfeasible compositions from the output space. Our model, Knowledge-Guided Simple Primitives (KG-SP), achieves state of the art in both OW-CZSL and pCZSL.
  arXiv  Detail & Related papers  (2022-05-13T17:18:15Z)
• Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition [101.59989523028264]
  Few-Shot image classification aims to utilize pretrained knowledge learned from a large-scale dataset to tackle a series of downstream classification tasks. We propose COSOC, a novel Few-Shot Learning framework, to automatically figure out foreground objects at both pretraining and evaluation stage.
  arXiv  Detail & Related papers  (2021-07-16T07:46:41Z)
• COMPAS: Representation Learning with Compositional Part Sharing for Few-Shot Classification [10.718573053194742]
  Few-shot image classification consists of two consecutive learning processes. Inspired by the compositional representation of objects in humans, we train a neural network architecture that explicitly represents objects as a set of parts. We demonstrate the value of our compositional learning framework for a few-shot classification using miniImageNet, tieredImageNet, CIFAR-FS, and FC100.
  arXiv  Detail & Related papers  (2021-01-28T09:16:21Z)
• Generative Multi-Label Zero-Shot Learning [136.17594611722285]
  Multi-label zero-shot learning strives to classify images into multiple unseen categories for which no data is available during training. Our work is the first to tackle the problem of multi-label feature in the (generalized) zero-shot setting. Our cross-level fusion-based generative approach outperforms the state-of-the-art on all three datasets.
  arXiv  Detail & Related papers  (2021-01-27T18:56:46Z)
• Learning the Compositional Visual Coherence for Complementary Recommendations [62.60648815930101]
  Complementary recommendations aim at providing users product suggestions that are supplementary and compatible with their obtained items. We propose a novel Content Attentive Neural Network (CANN) to model the comprehensive compositional coherence on both global contents and semantic contents.
  arXiv  Detail & Related papers  (2020-06-08T06:57:18Z)
• Compositional Embeddings for Multi-Label One-Shot Learning [30.748605784254355]
  We present a compositional embedding framework that infers not just a single class per input image, but a set of classes, in the setting of one-shot learning. Experiments on the OmniGlot, Open Images, and COCO datasets show that the proposed compositional embedding models outperform existing embedding methods.
  arXiv  Detail & Related papers  (2020-02-11T03:54:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.