GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

• URL: http://arxiv.org/abs/2210.02025v1
• Date: Wed, 5 Oct 2022 05:20:49 GMT
• Title: GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models
• Authors: Chen Liang, Wenguan Wang, Jiaxu Miao, Yi Yang
• Abstract summary: We propose a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature,class) With a variety of segmentation architectures and backbones, GMMSeg outperforms the discriminative counterparts on closed-set datasets. GMMSeg even performs well on open-world datasets.
• Score: 74.0430727476634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prevalent semantic segmentation solutions are, in essence, a dense discriminative classifier of p(class|pixel feature). Though straightforward, this de facto paradigm neglects the underlying data distribution p(pixel feature|class), and struggles to identify out-of-distribution data. Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature,class). For each class, GMMSeg builds Gaussian Mixture Models (GMMs) via Expectation-Maximization (EM), so as to capture class-conditional densities. Meanwhile, the deep dense representation is end-to-end trained in a discriminative manner, i.e., maximizing p(class|pixel feature). This endows GMMSeg with the strengths of both generative and discriminative models. With a variety of segmentation architectures and backbones, GMMSeg outperforms the discriminative counterparts on three closed-set datasets. More impressively, without any modification, GMMSeg even performs well on open-world datasets. We believe this work brings fundamental insights into the related fields.

Related papers

• ProtoGMM: Multi-prototype Gaussian-Mixture-based Domain Adaptation Model for Semantic Segmentation [0.8213829427624407]
  Domain adaptive semantic segmentation aims to generate accurate and dense predictions for an unlabeled target domain. We propose the ProtoGMM model, which incorporates the GMM into contrastive losses to perform guided contrastive learning. To achieve increased intra-class semantic similarity, decreased inter-class similarity, and domain alignment between the source and target domains, we employ multi-prototype contrastive learning.
  arXiv  Detail & Related papers  (2024-06-27T14:50:50Z)
• Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
  We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-04-26T06:00:27Z)
• Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning [56.45977379288308]
  Masked autoencoder (MAE) learns to reconstruct masked graph edges or node features. Contrastive Learning (CL) maximizes the similarity between augmented views of the same graph. We propose graph contrastive masked autoencoder (GCMAE) framework to unify MAE and CL.
  arXiv  Detail & Related papers  (2023-10-24T05:06:06Z)
• HGFormer: Hierarchical Grouping Transformer for Domain Generalized Semantic Segmentation [113.6560373226501]
  This work studies semantic segmentation under the domain generalization setting. We propose a novel hierarchical grouping transformer (HGFormer) to explicitly group pixels to form part-level masks and then whole-level masks. Experiments show that HGFormer yields more robust semantic segmentation results than per-pixel classification methods and flat grouping transformers.
  arXiv  Detail & Related papers  (2023-05-22T13:33:41Z)
• A Hybrid of Generative and Discriminative Models Based on the Gaussian-coupled Softmax Layer [5.33024001730262]
  We propose a method to train a hybrid of discriminative and generative models in a single neural network. We demonstrate that the proposed hybrid model can be applied to semi-supervised learning and confidence calibration.
  arXiv  Detail & Related papers  (2023-05-10T05:48:22Z)
• Multi-dataset Pretraining: A Unified Model for Semantic Segmentation [97.61605021985062]
  We propose a unified framework, termed as Multi-Dataset Pretraining, to take full advantage of the fragmented annotations of different datasets. This is achieved by first pretraining the network via the proposed pixel-to-prototype contrastive loss over multiple datasets. In order to better model the relationship among images and classes from different datasets, we extend the pixel level embeddings via cross dataset mixing.
  arXiv  Detail & Related papers  (2021-06-08T06:13:11Z)
• Cauchy-Schwarz Regularized Autoencoder [68.80569889599434]
  Variational autoencoders (VAE) are a powerful and widely-used class of generative models. We introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. Our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
  arXiv  Detail & Related papers  (2021-01-06T17:36:26Z)
• Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More [6.89001867562902]
  Max-Mahalanobis (MMC) can be trained discriminatively, generatively, or jointly for image classification and generation. We show that our Generative MMC (GMMC) can be trained discriminatively, generatively, or jointly for image classification and generation.
  arXiv  Detail & Related papers  (2021-01-01T00:42:04Z)
• Mixture of Conditional Gaussian Graphical Models for unlabelled heterogeneous populations in the presence of co-factors [0.0]
  Conditional correlation networks, within Gaussian Graphical Models (GGM), are widely used to describe the direct interactions between the components of a random vector. In this article, we propose a Mixture of Conditional GGM (CGGM) that subtracts the heterogeneous effects of the co-features to regroup the data points into sub-population corresponding clusters.
  arXiv  Detail & Related papers  (2020-06-19T11:57:30Z)
• Handling missing data in model-based clustering [0.0]
  We propose two methods to fit Gaussian mixtures in the presence of missing data. Both methods use a variant of the Monte Carlo Expectation-Maximisation algorithm for data augmentation. We show that the proposed methods outperform the multiple imputation approach, both in terms of clusters identification and density estimation.
  arXiv  Detail & Related papers  (2020-06-04T15:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.