Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition

• URL: http://arxiv.org/abs/2009.09450v1
• Date: Sun, 20 Sep 2020 15:05:29 GMT
• Title: Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition
• Authors: Tianshui Chen, Liang Lin, Riquan Chen, Xiaolu Hui, and Hefeng Wu
• Abstract summary: KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks. It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features. It exploits a graph propagation network to explore graph node interactions.
• Score: 75.44233392355711
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recognizing multiple labels of an image is a practical yet challenging task, and remarkable progress has been achieved by searching for semantic regions and exploiting label dependencies. However, current works utilize RNN/LSTM to implicitly capture sequential region/label dependencies, which cannot fully explore mutual interactions among the semantic regions/labels and do not explicitly integrate label co-occurrences. In addition, these works require large amounts of training samples for each category, and they are unable to generalize to novel categories with limited samples. To address these issues, we propose a knowledge-guided graph routing (KGGR) framework, which unifies prior knowledge of statistical label correlations with deep neural networks. The framework exploits prior knowledge to guide adaptive information propagation among different categories to facilitate multi-label analysis and reduce the dependency of training samples. Specifically, it first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features to initialize the graph, and it exploits a graph propagation network to explore graph node interactions, enabling learning contextualized image feature representations. Moreover, we initialize each graph node with the classifier weights for the corresponding label and apply another propagation network to transfer node messages through the graph. In this way, it can facilitate exploiting the information of correlated labels to help train better classifiers. We conduct extensive experiments on the traditional multi-label image recognition (MLR) and multi-label few-shot learning (ML-FSL) tasks and show that our KGGR framework outperforms the current state-of-the-art methods by sizable margins on the public benchmarks.

Related papers

• KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification [75.95647590619929]
  Zero-Shot Node Classification (ZNC) has been an emerging and crucial task in graph data analysis. We propose a Knowledge-Aware Multi-Faceted framework (KMF) that enhances the richness of label semantics. A novel geometric constraint is developed to alleviate the problem of prototype drift caused by node information aggregation.
  arXiv  Detail & Related papers  (2023-08-15T02:38:08Z)
• Multi-Label Continual Learning using Augmented Graph Convolutional Network [7.115602040521868]
  Multi-Label Continual Learning builds a class-incremental framework in a sequential multi-label image recognition data stream. The study proposes an Augmented Graph Convolutional Network (AGCN++) that can construct the cross-task label relationships in MLCL. The proposed method is evaluated using two multi-label image benchmarks.
  arXiv  Detail & Related papers  (2022-11-27T08:40:19Z)
• AGCN: Augmented Graph Convolutional Network for Lifelong Multi-label Image Recognition [8.06616666179388]
  The Lifelong Multi-Label (LML) image recognition builds an online class-incremental classifier in a sequential multi-label image recognition data stream. The key challenges of LML image recognition are the construction of label relationships on Partial Labels of training data and the Catastrophic Forgetting on old classes. We propose an Augmented Graph Convolutional Network (AGCN) model that can construct the label relationships across the sequential recognition tasks.
  arXiv  Detail & Related papers  (2022-03-10T18:37:56Z)
• Graph Attention Transformer Network for Multi-Label Image Classification [50.0297353509294]
  We propose a general framework for multi-label image classification that can effectively mine complex inter-label relationships. Our proposed methods can achieve state-of-the-art performance on three datasets.
  arXiv  Detail & Related papers  (2022-03-08T12:39:05Z)
• Structured Semantic Transfer for Multi-Label Recognition with Partial Labels [85.6967666661044]
  We propose a structured semantic transfer (SST) framework that enables training multi-label recognition models with partial labels. The framework consists of two complementary transfer modules that explore within-image and cross-image semantic correlations. Experiments on the Microsoft COCO, Visual Genome and Pascal VOC datasets show that the proposed SST framework obtains superior performance over current state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-12-21T02:15:01Z)
• Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition [53.17837649440601]
  We propose an Attention-Driven Dynamic Graph Convolutional Network (ADD-GCN) to dynamically generate a specific graph for each image. Experiments on public multi-label benchmarks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2020-12-05T10:10:12Z)
• Instance-Aware Graph Convolutional Network for Multi-Label Classification [55.131166957803345]
  Graph convolutional neural network (GCN) has effectively boosted the multi-label image recognition task. We propose an instance-aware graph convolutional neural network (IA-GCN) framework for multi-label classification.
  arXiv  Detail & Related papers  (2020-08-19T12:49:28Z)
• Multi-label Zero-shot Classification by Learning to Transfer from External Knowledge [36.04579549557464]
  Multi-label zero-shot classification aims to predict multiple unseen class labels for an input image. This paper introduces a novel multi-label zero-shot classification framework by learning to transfer from external knowledge.
  arXiv  Detail & Related papers  (2020-07-30T17:26:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.