Semantic-embedded Similarity Prototype for Scene Recognition

• URL: http://arxiv.org/abs/2308.05896v3
• Date: Mon, 20 May 2024 01:59:46 GMT
• Title: Semantic-embedded Similarity Prototype for Scene Recognition
• Authors: Chuanxin Song, Hanbo Wu, Xin Ma, Yibin Li,
• Abstract summary: This paper proposes a semantic knowledge-based similarity prototype. It can help the scene recognition network achieve superior accuracy without increasing the computational cost in practice. Our similarity prototype enhances the performance of existing networks, all while avoiding any additional computational burden in practical deployments.
• Score: 12.236534954126155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the high inter-class similarity caused by the complex composition and the co-existing objects across scenes, numerous studies have explored object semantic knowledge within scenes to improve scene recognition. However, a resulting challenge emerges as object information extraction techniques require heavy computational costs, thereby burdening the network considerably. This limitation often renders object-assisted approaches incompatible with edge devices in practical deployment. In contrast, this paper proposes a semantic knowledge-based similarity prototype, which can help the scene recognition network achieve superior accuracy without increasing the computational cost in practice. It is simple and can be plug-and-played into existing pipelines. More specifically, a statistical strategy is introduced to depict semantic knowledge in scenes as class-level semantic representations. These representations are used to explore correlations between scene classes, ultimately constructing a similarity prototype. Furthermore, we propose to leverage the similarity prototype to support network training from the perspective of Gradient Label Softening and Batch-level Contrastive Loss, respectively. Comprehensive evaluations on multiple benchmarks show that our similarity prototype enhances the performance of existing networks, all while avoiding any additional computational burden in practical deployments. Code and the statistical similarity prototype will be available at https://github.com/ChuanxinSong/SimilarityPrototype

Related papers

• Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation [22.591512454923883]
  We argue that the knowledge bias between instances and contexts affects the capability of the prototype to sufficiently understand instance semantics. Inspired by prototype learning theory, we propose leveraging prototype awareness to capture diverse and fine-grained feature attributes of instances. We present a Context Prototype-Aware Learning (CPAL) strategy, which leverages semantic context to enrich instance comprehension.
  arXiv  Detail & Related papers  (2024-03-12T13:11:58Z)
• Adaptive Prototypical Networks [2.964978357715084]
  Prototypical network for Few shot learning tries to learn an embedding function in the encoder that embeds images with similar features close to one another. We propose an approach that intuitively pushes the embeddings of each of the classes away from the others in the meta-testing phase. This is achieved by training the encoder network for classification using the support set samples and labels of the new task.
  arXiv  Detail & Related papers  (2022-11-22T18:45:58Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• Semantic Representation and Dependency Learning for Multi-Label Image Recognition [76.52120002993728]
  We propose a novel and effective semantic representation and dependency learning (SRDL) framework to learn category-specific semantic representation for each category. Specifically, we design a category-specific attentional regions (CAR) module to generate channel/spatial-wise attention matrices to guide model. We also design an object erasing (OE) module to implicitly learn semantic dependency among categories by erasing semantic-aware regions.
  arXiv  Detail & Related papers  (2022-04-08T00:55:15Z)
• Dense Semantic Contrast for Self-Supervised Visual Representation Learning [12.636783522731392]
  We present Dense Semantic Contrast (DSC) for modeling semantic category decision boundaries at a dense level. We propose a dense cross-image semantic contrastive learning framework for multi-granularity representation learning. Experimental results show that our DSC model outperforms state-of-the-art methods when transferring to downstream dense prediction tasks.
  arXiv  Detail & Related papers  (2021-09-16T07:04:05Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)
• Toward Scalable and Unified Example-based Explanation and Outlier Detection [128.23117182137418]
  We argue for a broader adoption of prototype-based student networks capable of providing an example-based explanation for their prediction. We show that our prototype-based networks beyond similarity kernels deliver meaningful explanations and promising outlier detection results without compromising classification accuracy.
  arXiv  Detail & Related papers  (2020-11-11T05:58:17Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)
• Region Comparison Network for Interpretable Few-shot Image Classification [97.97902360117368]
  Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes. We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works. We also present a new way to generalize the interpretability from the level of tasks to categories.
  arXiv  Detail & Related papers  (2020-09-08T07:29:05Z)
• Contextual Encoder-Decoder Network for Visual Saliency Prediction [42.047816176307066]
  We propose an approach based on a convolutional neural network pre-trained on a large-scale image classification task. We combine the resulting representations with global scene information for accurately predicting visual saliency. Compared to state of the art approaches, the network is based on a lightweight image classification backbone.
  arXiv  Detail & Related papers  (2019-02-18T16:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.