Instance-wise Uncertainty for Class Imbalance in Semantic Segmentation

• URL: http://arxiv.org/abs/2407.12609v1
• Date: Wed, 17 Jul 2024 14:38:32 GMT
• Title: Instance-wise Uncertainty for Class Imbalance in Semantic Segmentation
• Authors: Luís Almeida, Inês Dutra, Francesco Renna,
• Abstract summary: State of the art methods increasingly rely on deep learning models, known to incorrectly estimate uncertainty and be overconfident in predictions. This is particularly problematic in semantic segmentation due to inherent class imbalance. A novel training methodology specifically designed for semantic segmentation is presented.
• Score: 4.147659576493158
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Semantic segmentation is a fundamental computer vision task with a vast number of applications. State of the art methods increasingly rely on deep learning models, known to incorrectly estimate uncertainty and being overconfident in predictions, especially in data not seen during training. This is particularly problematic in semantic segmentation due to inherent class imbalance. Popular uncertainty quantification approaches are task-agnostic and fail to leverage spatial pixel correlations in uncertainty estimates, crucial in this task. In this work, a novel training methodology specifically designed for semantic segmentation is presented. Training samples are weighted by instance-wise uncertainty masks computed by an ensemble. This is shown to increase performance on minority classes, boost model generalization and robustness to domain-shift when compared to using the inverse of class proportions or no class weights at all. This method addresses the challenges of class imbalance and uncertainty estimation in semantic segmentation, potentially enhancing model performance and reliability across various applications.

Related papers

• Tendency-driven Mutual Exclusivity for Weakly Supervised Incremental Semantic Segmentation [56.1776710527814]
  Weakly Incremental Learning for Semantic (WILSS) leverages a pre-trained segmentation model to segment new classes using cost-effective and readily available image-level labels. A prevailing way to solve WILSS is the generation of seed areas for each new class, serving as a form of pixel-level supervision. We propose an innovative, tendency-driven relationship of mutual exclusivity, meticulously tailored to govern the behavior of the seed areas.
  arXiv  Detail & Related papers  (2024-04-18T08:23:24Z)
• Towards Better Certified Segmentation via Diffusion Models [62.21617614504225]
  segmentation models can be vulnerable to adversarial perturbations, which hinders their use in critical-decision systems like healthcare or autonomous driving. Recently, randomized smoothing has been proposed to certify segmentation predictions by adding Gaussian noise to the input to obtain theoretical guarantees. In this paper, we address the problem of certifying segmentation prediction using a combination of randomized smoothing and diffusion models.
  arXiv  Detail & Related papers  (2023-06-16T16:30:39Z)
• Modeling Uncertain Feature Representation for Domain Generalization [49.129544670700525]
  We show that our method consistently improves the network generalization ability on multiple vision tasks. Our methods are simple yet effective and can be readily integrated into networks without additional trainable parameters or loss constraints.
  arXiv  Detail & Related papers  (2023-01-16T14:25:02Z)
• Post-hoc Uncertainty Learning using a Dirichlet Meta-Model [28.522673618527417]
  We propose a novel Bayesian meta-model to augment pre-trained models with better uncertainty quantification abilities. Our proposed method requires no additional training data and is flexible enough to quantify different uncertainties. We demonstrate our proposed meta-model approach's flexibility and superior empirical performance on these applications.
  arXiv  Detail & Related papers  (2022-12-14T17:34:11Z)
• Instance-specific and Model-adaptive Supervision for Semi-supervised Semantic Segmentation [49.82432158155329]
  We propose an instance-specific and model-adaptive supervision for semi-supervised semantic segmentation, named iMAS. iMAS learns from unlabeled instances progressively by weighing their corresponding consistency losses based on the evaluated hardness.
  arXiv  Detail & Related papers  (2022-11-21T10:37:28Z)
• Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation [0.42008820076301906]
  Uncertainty quantification in automated image analysis is highly desired in many applications. Current uncertainty quantification approaches do not scale well in high-dimensional real-world problems. We propose a scalable and intuitive framework to calibrate ensembles of deep learning models to produce uncertainty quantification measurements.
  arXiv  Detail & Related papers  (2022-09-20T09:09:48Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Analyzing Overfitting under Class Imbalance in Neural Networks for Image Segmentation [19.259574003403998]
  In image segmentation neural networks may overfit to the foreground samples from small structures. In this study, we provide new insights on the problem of overfitting under class imbalance by inspecting the network behavior.
  arXiv  Detail & Related papers  (2021-02-20T14:57:58Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)
• Probabilistic Deep Learning for Instance Segmentation [9.62543698736491]
  We propose a generic method to obtain model-inherent uncertainty estimates within proposal-free instance segmentation models. We evaluate our method on the BBBC010 C. elegans dataset, where it yields competitive performance.
  arXiv  Detail & Related papers  (2020-08-24T19:51:48Z)
• Domain segmentation and adjustment for generalized zero-shot learning [22.933463036413624]
  In zero-shot learning, synthesizing unseen data with generative models has been the most popular method to address the imbalance of training data between seen and unseen classes. We argue that synthesizing unseen data may not be an ideal approach for addressing the domain shift caused by the imbalance of the training data. In this paper, we propose to realize the generalized zero-shot recognition in different domains.
  arXiv  Detail & Related papers  (2020-02-01T15:00:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.