Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representations

• URL: http://arxiv.org/abs/2210.14670v1
• Date: Wed, 26 Oct 2022 12:47:29 GMT
• Title: Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representations
• Authors: Haoyu Xie, Changqi Wang, Mingkai Zheng, Minjing Dong, Shan You, Chang Xu
• Abstract summary: We propose a Probabilistic Representation Contrastive Learning framework to improve representation quality. We define pixel-wise representations from a new perspective of probability theory. We also propose to regularize the distribution variance to enhance the reliability of representations.
• Score: 30.672426195148496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent breakthroughs in semi-supervised semantic segmentation have been developed through contrastive learning. In prevalent pixel-wise contrastive learning solutions, the model maps pixels to deterministic representations and regularizes them in the latent space. However, there exist inaccurate pseudo-labels which map the ambiguous representations of pixels to the wrong classes due to the limited cognitive ability of the model. In this paper, we define pixel-wise representations from a new perspective of probability theory and propose a Probabilistic Representation Contrastive Learning (PRCL) framework that improves representation quality by taking its probability into consideration. Through modeling the mapping from pixels to representations as the probability via multivariate Gaussian distributions, we can tune the contribution of the ambiguous representations to tolerate the risk of inaccurate pseudo-labels. Furthermore, we define prototypes in the form of distributions, which indicates the confidence of a class, while the point prototype cannot. Moreover, we propose to regularize the distribution variance to enhance the reliability of representations. Taking advantage of these benefits, high-quality feature representations can be derived in the latent space, thereby the performance of semantic segmentation can be further improved. We conduct sufficient experiment to evaluate PRCL on Pascal VOC and CityScapes. The comparisons with state-of-the-art approaches demonstrate the superiority of proposed PRCL.

Related papers

• Reducing Semantic Ambiguity In Domain Adaptive Semantic Segmentation Via Probabilistic Prototypical Pixel Contrast [7.092718945468069]
  Domain adaptation aims to reduce the model degradation on the target domain caused by the domain shift between the source and target domains. Probabilistic proto-typical pixel contrast (PPPC) is a universal adaptation framework that models each pixel embedding as a probability. PPPC not only helps to address ambiguity at the pixel level, yielding discriminative representations but also significant improvements in both synthetic-to-real and day-to-night adaptation tasks.
  arXiv  Detail & Related papers  (2024-09-27T08:25:03Z)
• Efficient Fairness-Performance Pareto Front Computation [51.558848491038916]
  We show that optimal fair representations possess several useful structural properties. We then show that these approxing problems can be solved efficiently via concave programming methods.
  arXiv  Detail & Related papers  (2024-09-26T08:46:48Z)
• FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms [60.195642571004804]
  We propose FlowSDF, an image-guided conditional flow matching framework to represent the signed distance function (SDF) By learning a vector field that is directly related to the probability path of a conditional distribution of SDFs, we can accurately sample from the distribution of segmentation masks.
  arXiv  Detail & Related papers  (2024-05-28T11:47:12Z)
• PRCL: Probabilistic Representation Contrastive Learning for Semi-Supervised Semantic Segmentation [15.869077228828303]
  We propose a robust contrastive-based S4 framework, termed the Probabilistic Representation Contrastive Learning (PRCL) framework to enhance the robustness of the unsupervised training process.
  arXiv  Detail & Related papers  (2024-02-28T07:10:37Z)
• Flow Factorized Representation Learning [109.51947536586677]
  We introduce a generative model which specifies a distinct set of latent probability paths that define different input transformations. We show that our model achieves higher likelihoods on standard representation learning benchmarks while simultaneously being closer to approximately equivariant models.
  arXiv  Detail & Related papers  (2023-09-22T20:15:37Z)
• Learning Disentangled Discrete Representations [22.5004558029479]
  We show the relationship between discrete latent spaces and disentangled representations by replacing the standard Gaussian variational autoencoder with a tailored categorical variational autoencoder. We provide both analytical and empirical findings that demonstrate the advantages of discrete VAEs for learning disentangled representations.
  arXiv  Detail & Related papers  (2023-07-26T12:29:58Z)
• Score-Based Generative Models for Medical Image Segmentation using Signed Distance Functions [11.137438870686026]
  We propose a conditional score-based generative modeling framework to represent the signed distance function (SDF) The advantage of leveraging the SDF is a more natural distortion when compared to that of binary masks.
  arXiv  Detail & Related papers  (2023-03-10T14:55:35Z)
• Uncertainty-based Cross-Modal Retrieval with Probabilistic Representations [18.560958487332265]
  Probabilistic embeddings have proven useful for capturing polysemous word meanings, as well as ambiguity in image matching. We propose a simple approach that replaces the standard vector point embeddings in extant image-text matching models with probabilistic distributions that are parametrically learned.
  arXiv  Detail & Related papers  (2022-04-20T07:24:20Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• PSD Representations for Effective Probability Models [117.35298398434628]
  We show that a recently proposed class of positive semi-definite (PSD) models for non-negative functions is particularly suited to this end. We characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees. Our results open the way to applications of PSD models to density estimation, decision theory and inference.
  arXiv  Detail & Related papers  (2021-06-30T15:13:39Z)
• Attentional Prototype Inference for Few-Shot Segmentation [128.45753577331422]
  We propose attentional prototype inference (API), a probabilistic latent variable framework for few-shot segmentation. We define a global latent variable to represent the prototype of each object category, which we model as a probabilistic distribution. We conduct extensive experiments on four benchmarks, where our proposal obtains at least competitive and often better performance than state-of-the-art prototype-based methods.
  arXiv  Detail & Related papers  (2021-05-14T06:58:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.