Semantic-Aware Representation Learning for Multi-label Image Classification

• URL: http://arxiv.org/abs/2507.14918v1
• Date: Sun, 20 Jul 2025 11:15:24 GMT
• Title: Semantic-Aware Representation Learning for Multi-label Image Classification
• Authors: Ren-Dong Xie, Zhi-Fen He, Bo Li, Bin Liu, Jin-Yan Hu,
• Abstract summary: This paper proposes a Semantic-Aware Representation Learning (SARL) for multi-label image classification.<n>First, a label semantic-related feature learning module is utilized to extract semantic-related features.<n>Second, an optimal transport-based attention mechanism is designed to obtain semantically aligned image representation.
• Score: 6.444512435220748
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-label image classification, an important research area in computer vision, focuses on identifying multiple labels or concepts within an image. Existing approaches often employ attention mechanisms or graph convolutional networks (GCNs) to learn image representation. However, this representation may contain noise and may not locate objects precisely. Therefore, this paper proposes a Semantic-Aware Representation Learning (SARL) for multi-label image classification. First, a label semantic-related feature learning module is utilized to extract semantic-related features. Then, an optimal transport-based attention mechanism is designed to obtain semantically aligned image representation. Finally, a regional score aggregation strategy is used for multi-label prediction. Experimental results on two benchmark datasets, PASCAL VOC 2007 and MS-COCO, demonstrate the superiority of SARL over existing methods.

Related papers

• Semantic-guided Representation Learning for Multi-Label Recognition [13.046479112800608]
  Multi-label Recognition (MLR) involves assigning multiple labels to each data instance in an image.<n>Recent Vision and Language Pre-training methods have made significant progress in tackling zero-shot MLR tasks.<n>We introduce a Semantic-guided Representation Learning approach (SigRL) that enables the model to learn effective visual and textual representations.
  arXiv  Detail & Related papers  (2025-04-04T08:15:08Z)
• Multi-Label Self-Supervised Learning with Scene Images [21.549234013998255]
  This paper shows that quality image representations can be learned by treating scene/multi-label image SSL simply as a multi-label classification problem. The proposed method is named Multi-Label Self-supervised learning (MLS)
  arXiv  Detail & Related papers  (2023-08-07T04:04:22Z)
• PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification [48.929583521641526]
  Multi-label image classification is a prediction task that aims to identify more than one label from a given image. This paper introduces the conditional transport theory to bridge the acknowledged gap. We find that by formulating the multi-label classification as a CT problem, we can exploit the interactions between the image and label efficiently.
  arXiv  Detail & Related papers  (2023-07-18T08:37:37Z)
• Dual-Perspective Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels [70.36722026729859]
  We propose a dual-perspective semantic-aware representation blending (DSRB) that blends multi-granularity category-specific semantic representation across different images. The proposed DS consistently outperforms current state-of-the-art algorithms on all proportion label settings.
  arXiv  Detail & Related papers  (2022-05-26T00:33:44Z)
• Semantic Representation and Dependency Learning for Multi-Label Image Recognition [76.52120002993728]
  We propose a novel and effective semantic representation and dependency learning (SRDL) framework to learn category-specific semantic representation for each category. Specifically, we design a category-specific attentional regions (CAR) module to generate channel/spatial-wise attention matrices to guide model. We also design an object erasing (OE) module to implicitly learn semantic dependency among categories by erasing semantic-aware regions.
  arXiv  Detail & Related papers  (2022-04-08T00:55:15Z)
• Multi-layered Semantic Representation Network for Multi-label Image Classification [8.17894017454724]
  Multi-label image classification (MLIC) is a fundamental and practical task, which aims to assign multiple possible labels to an image. In recent years, many deep convolutional neural network (CNN) based approaches have been proposed which model label correlations. This paper advances this research direction by improving the modeling of label correlations and the learning of semantic representations.
  arXiv  Detail & Related papers  (2021-06-22T08:04:22Z)
• Semantic Diversity Learning for Zero-Shot Multi-label Classification [14.480713752871523]
  This study introduces an end-to-end model training for multi-label zero-shot learning. We propose to use an embedding matrix having principal embedding vectors trained using a tailored loss function. In addition, during training, we suggest up-weighting in the loss function image samples presenting higher semantic diversity to encourage the diversity of the embedding matrix.
  arXiv  Detail & Related papers  (2021-05-12T19:39:07Z)
• Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition [75.44233392355711]
  KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks. It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features. It exploits a graph propagation network to explore graph node interactions.
  arXiv  Detail & Related papers  (2020-09-20T15:05:29Z)
• Zero-Shot Recognition through Image-Guided Semantic Classification [9.291055558504588]
  We present a new embedding-based framework for zero-shot learning (ZSL) Motivated by the binary relevance method for multi-label classification, we propose to inversely learn the mapping between an image and a semantic classifier. IGSC is conceptually simple and can be realized by a slight enhancement of an existing deep architecture for classification.
  arXiv  Detail & Related papers  (2020-07-23T06:22:40Z)
• Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation [128.03739769844736]
  Two neural co-attentions are incorporated into the classifier to capture cross-image semantic similarities and differences. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference. Our algorithm sets new state-of-the-arts on all these settings, demonstrating well its efficacy and generalizability.
  arXiv  Detail & Related papers  (2020-07-03T21:53:46Z)
• Hierarchical Image Classification using Entailment Cone Embeddings [68.82490011036263]
  We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier. We empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance.
  arXiv  Detail & Related papers  (2020-04-02T10:22:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.