Contextrast: Contextual Contrastive Learning for Semantic Segmentation

• URL: http://arxiv.org/abs/2404.10633v2
• Date: Tue, 08 Oct 2024 02:12:27 GMT
• Title: Contextrast: Contextual Contrastive Learning for Semantic Segmentation
• Authors: Changki Sung, Wanhee Kim, Jungho An, Wooju Lee, Hyungtae Lim, Hyun Myung,
• Abstract summary: We propose Contextrast, a contrastive learning-based semantic segmentation method. Our proposed method comprises two parts: a) contextual contrastive learning (CCL) and b) boundary-aware negative sampling. We demonstrate that our Contextrast substantially enhances the performance of semantic segmentation networks.
• Score: 9.051352746190448
• License:
• Abstract: Despite great improvements in semantic segmentation, challenges persist because of the lack of local/global contexts and the relationship between them. In this paper, we propose Contextrast, a contrastive learning-based semantic segmentation method that allows to capture local/global contexts and comprehend their relationships. Our proposed method comprises two parts: a) contextual contrastive learning (CCL) and b) boundary-aware negative (BANE) sampling. Contextual contrastive learning obtains local/global context from multi-scale feature aggregation and inter/intra-relationship of features for better discrimination capabilities. Meanwhile, BANE sampling selects embedding features along the boundaries of incorrectly predicted regions to employ them as harder negative samples on our contrastive learning, resolving segmentation issues along the boundary region by exploiting fine-grained details. We demonstrate that our Contextrast substantially enhances the performance of semantic segmentation networks, outperforming state-of-the-art contrastive learning approaches on diverse public datasets, e.g. Cityscapes, CamVid, PASCAL-C, COCO-Stuff, and ADE20K, without an increase in computational cost during inference.

Related papers

• Text-Video Retrieval with Global-Local Semantic Consistent Learning [122.15339128463715]
  We propose a simple yet effective method, Global-Local Semantic Consistent Learning (GLSCL) GLSCL capitalizes on latent shared semantics across modalities for text-video retrieval. Our method achieves comparable performance with SOTA as well as being nearly 220 times faster in terms of computational cost.
  arXiv  Detail & Related papers  (2024-05-21T11:59:36Z)
• Multi-Grained Cross-modal Alignment for Learning Open-vocabulary Semantic Segmentation from Text Supervision [23.931443799102663]
  We introduce a Multi-Grained Cross-modal Alignment (MGCA) framework to bridge the granularity gap without any dense annotations. Specifically, MGCA constructs pseudo multi-granular semantic correspondences upon image-text pairs. Our method achieves significant advancements over state-of-the-art methods, demonstrating its effectiveness and efficiency.
  arXiv  Detail & Related papers  (2024-03-06T13:43:36Z)
• Associating Spatially-Consistent Grouping with Text-supervised Semantic Segmentation [117.36746226803993]
  We introduce self-supervised spatially-consistent grouping with text-supervised semantic segmentation. Considering the part-like grouped results, we further adapt a text-supervised model from image-level to region-level recognition. Our method achieves 59.2% mIoU and 32.4% mIoU on Pascal VOC and Pascal Context benchmarks.
  arXiv  Detail & Related papers  (2023-04-03T16:24:39Z)
• Context Label Learning: Improving Background Class Representations in Semantic Segmentation [23.79946807540805]
  We find that neural networks trained with heterogeneous background struggle to map the corresponding contextual samples to compact clusters in feature space. We propose context label learning (CoLab) to improve the context representations by decomposing the background class into several subclasses. The results demonstrate that CoLab can guide the segmentation model to map the logits of background samples away from the decision boundary.
  arXiv  Detail & Related papers  (2022-12-16T11:52:15Z)
• Regional Semantic Contrast and Aggregation for Weakly Supervised Semantic Segmentation [25.231470587575238]
  We propose regional semantic contrast and aggregation (RCA) for learning semantic segmentation. RCA is equipped with a regional memory bank to store massive, diverse object patterns appearing in training data. RCA earns a strong capability of fine-grained semantic understanding, and eventually establishes new state-of-the-art results on two popular benchmarks.
  arXiv  Detail & Related papers  (2022-03-17T23:29:03Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Boundary Guided Context Aggregation for Semantic Segmentation [23.709865471981313]
  We exploit boundary as a significant guidance for context aggregation to promote the overall semantic understanding of an image. We conduct extensive experiments on the Cityscapes and ADE20K databases, and comparable results are achieved with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-27T17:04:38Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• CTNet: Context-based Tandem Network for Semantic Segmentation [77.4337867789772]
  This work proposes a novel Context-based Tandem Network (CTNet) by interactively exploring the spatial contextual information and the channel contextual information. To further improve the performance of the learned representations for semantic segmentation, the results of the two context modules are adaptively integrated.
  arXiv  Detail & Related papers  (2021-04-20T07:33:11Z)
• Deep Clustering by Semantic Contrastive Learning [67.28140787010447]
  We introduce a novel variant called Semantic Contrastive Learning (SCL) It explores the characteristics of both conventional contrastive learning and deep clustering. It can amplify the strengths of contrastive learning and deep clustering in a unified approach.
  arXiv  Detail & Related papers  (2021-03-03T20:20:48Z)
• Contextual Diversity for Active Learning [9.546771465714876]
  Large datasets restrict the use of deep convolutional neural networks (CNNs) for many practical applications. We introduce the notion of contextual diversity that captures the confusion associated with spatially co-occurring classes. Our studies show clear advantages of using contextual diversity for active learning.
  arXiv  Detail & Related papers  (2020-08-13T07:04:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.