Multi-layered Semantic Representation Network for Multi-label Image Classification

• URL: http://arxiv.org/abs/2106.11596v1
• Date: Tue, 22 Jun 2021 08:04:22 GMT
• Title: Multi-layered Semantic Representation Network for Multi-label Image Classification
• Authors: Xiwen Qu, Hao Che, Jun Huang, Linchuan Xu, Xiao Zheng
• Abstract summary: Multi-label image classification (MLIC) is a fundamental and practical task, which aims to assign multiple possible labels to an image. In recent years, many deep convolutional neural network (CNN) based approaches have been proposed which model label correlations. This paper advances this research direction by improving the modeling of label correlations and the learning of semantic representations.
• Score: 8.17894017454724
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-label image classification (MLIC) is a fundamental and practical task, which aims to assign multiple possible labels to an image. In recent years, many deep convolutional neural network (CNN) based approaches have been proposed which model label correlations to discover semantics of labels and learn semantic representations of images. This paper advances this research direction by improving both the modeling of label correlations and the learning of semantic representations. On the one hand, besides the local semantics of each label, we propose to further explore global semantics shared by multiple labels. On the other hand, existing approaches mainly learn the semantic representations at the last convolutional layer of a CNN. But it has been noted that the image representations of different layers of CNN capture different levels or scales of features and have different discriminative abilities. We thus propose to learn semantic representations at multiple convolutional layers. To this end, this paper designs a Multi-layered Semantic Representation Network (MSRN) which discovers both local and global semantics of labels through modeling label correlations and utilizes the label semantics to guide the semantic representations learning at multiple layers through an attention mechanism. Extensive experiments on four benchmark datasets including VOC 2007, COCO, NUS-WIDE, and Apparel show a competitive performance of the proposed MSRN against state-of-the-art models.

Related papers

• Semantic-Aware Graph Matching Mechanism for Multi-Label Image Recognition [21.36538164675385]
  Multi-label image recognition aims to predict a set of labels that present in an image. In this paper, we treat each image as a bag of instances, and formulate the task of multi-label image recognition as an instance-label matching selection problem. We propose an innovative Semantic-aware Graph Matching framework for Multi-Label image recognition (ML-SGM)
  arXiv  Detail & Related papers  (2023-04-21T23:48:01Z)
• Dual-Perspective Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels [70.36722026729859]
  We propose a dual-perspective semantic-aware representation blending (DSRB) that blends multi-granularity category-specific semantic representation across different images. The proposed DS consistently outperforms current state-of-the-art algorithms on all proportion label settings.
  arXiv  Detail & Related papers  (2022-05-26T00:33:44Z)
• Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels [70.45813147115126]
  Multi-label image recognition with partial labels (MLR-PL) may greatly reduce the cost of annotation and thus facilitate large-scale MLR. We find that strong semantic correlations exist within each image and across different images. These correlations can help transfer the knowledge possessed by the known labels to retrieve the unknown labels.
  arXiv  Detail & Related papers  (2022-05-23T08:37:38Z)
• Structured Semantic Transfer for Multi-Label Recognition with Partial Labels [85.6967666661044]
  We propose a structured semantic transfer (SST) framework that enables training multi-label recognition models with partial labels. The framework consists of two complementary transfer modules that explore within-image and cross-image semantic correlations. Experiments on the Microsoft COCO, Visual Genome and Pascal VOC datasets show that the proposed SST framework obtains superior performance over current state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2021-12-21T02:15:01Z)
• Spatial-context-aware deep neural network for multi-class image classification [12.961070515143161]
  A spatial-context-aware deep neural network is proposed to predict labels taking into account both semantic and spatial information. This proposed framework is evaluated on Microsoft COCO and PASCAL VOC, two widely used benchmark datasets for image multi-labelling.
  arXiv  Detail & Related papers  (2021-11-24T06:36:10Z)
• Semantic Diversity Learning for Zero-Shot Multi-label Classification [14.480713752871523]
  This study introduces an end-to-end model training for multi-label zero-shot learning. We propose to use an embedding matrix having principal embedding vectors trained using a tailored loss function. In addition, during training, we suggest up-weighting in the loss function image samples presenting higher semantic diversity to encourage the diversity of the embedding matrix.
  arXiv  Detail & Related papers  (2021-05-12T19:39:07Z)
• Deep Semantic Dictionary Learning for Multi-label Image Classification [3.3989824361632337]
  We present an innovative path towards the solution of the multi-label image classification which considers it as a dictionary learning task. A novel end-to-end model named Deep Semantic Dictionary Learning (DSDL) is designed. Our codes and models have been released.
  arXiv  Detail & Related papers  (2020-12-23T06:22:47Z)
• Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition [75.44233392355711]
  KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks. It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features. It exploits a graph propagation network to explore graph node interactions.
  arXiv  Detail & Related papers  (2020-09-20T15:05:29Z)
• Instance-Aware Graph Convolutional Network for Multi-Label Classification [55.131166957803345]
  Graph convolutional neural network (GCN) has effectively boosted the multi-label image recognition task. We propose an instance-aware graph convolutional neural network (IA-GCN) framework for multi-label classification.
  arXiv  Detail & Related papers  (2020-08-19T12:49:28Z)
• Hierarchical Image Classification using Entailment Cone Embeddings [68.82490011036263]
  We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier. We empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance.
  arXiv  Detail & Related papers  (2020-04-02T10:22:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.