Uncertainty evaluation of segmentation models for Earth observation

• URL: http://arxiv.org/abs/2510.19586v1
• Date: Wed, 22 Oct 2025 13:39:28 GMT
• Title: Uncertainty evaluation of segmentation models for Earth observation
• Authors: Melanie Rey, Andriy Mnih, Maxim Neumann, Matt Overlan, Drew Purves,
• Abstract summary: This paper investigates methods for estimating uncertainty in semantic segmentation predictions derived from satellite imagery.<n>Our evaluation focuses on the practical utility of uncertainty measures, testing their ability to identify prediction errors and noise-corrupted input image regions.
• Score: 4.350621291554061
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates methods for estimating uncertainty in semantic segmentation predictions derived from satellite imagery. Estimating uncertainty for segmentation presents unique challenges compared to standard image classification, requiring scalable methods producing per-pixel estimates. While most research on this topic has focused on scene understanding or medical imaging, this work benchmarks existing methods specifically for remote sensing and Earth observation applications. Our evaluation focuses on the practical utility of uncertainty measures, testing their ability to identify prediction errors and noise-corrupted input image regions. Experiments are conducted on two remote sensing datasets, PASTIS and ForTy, selected for their differences in scale, geographic coverage, and label confidence. We perform an extensive evaluation featuring several models, such as Stochastic Segmentation Networks and ensembles, in combination with a number of neural architectures and uncertainty metrics. We make a number of practical recommendations based on our findings.

Related papers

• A Meaningful Perturbation Metric for Evaluating Explainability Methods [55.09730499143998]
  We introduce a novel approach, which harnesses image generation models to perform targeted perturbation.<n> Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity.<n>This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results.
  arXiv  Detail & Related papers  (2025-04-09T11:46:41Z)
• A Dataset for Semantic Segmentation in the Presence of Unknowns [49.795683850385956]
  Existing datasets allow evaluation of only knowns or unknowns - but not both.<n>We propose a novel anomaly segmentation dataset, ISSU, that features a diverse set of anomaly inputs from cluttered real-world environments.<n>The dataset is twice larger than existing anomaly segmentation datasets.
  arXiv  Detail & Related papers  (2025-03-28T10:31:01Z)
• PaRCE: Probabilistic and Reconstruction-based Competency Estimation for CNN-based Image Classification [0.10923877073891446]
  We develop a probabilistic and reconstruction-based competency estimation (PaRCE) method.<n>We find that our method can best distinguish between correctly classified, misclassified, and OOD samples with anomalous regions.<n>Our method generates interpretable scores that most reliably capture a holistic notion of perception model confidence.
  arXiv  Detail & Related papers  (2024-11-22T22:08:57Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Anatomically-aware Uncertainty for Semi-supervised Image Segmentation [12.175556059523863]
  Semi-supervised learning relaxes the need of large pixel-wise labeled datasets for image segmentation by leveraging unlabeled data. Uncertainty estimation methods rely on multiple inferences from the model predictions that must be computed for each training step. This work proposes a novel method to estimate segmentation uncertainty by leveraging global information from the segmentation masks.
  arXiv  Detail & Related papers  (2023-10-24T18:03:07Z)
• Uncertainty Estimation in Instance Segmentation with Star-convex Shapes [4.197316670989004]
  Deep neural network-based algorithms often exhibit incorrect predictions with unwarranted confidence levels. Our study addresses the challenge of estimating spatial certainty with the location of instances with star- shapes. Our study demonstrates that combining fractional certainty estimation over individual certainty scores is an effective strategy.
  arXiv  Detail & Related papers  (2023-09-19T10:49:33Z)
• Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise [62.997667081978825]
  In high-risk environments, deep learning models need to be able to judge their uncertainty and reject inputs when there is a significant chance of misclassification. We conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise.
  arXiv  Detail & Related papers  (2023-01-03T11:34:36Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• Probabilistic Deep Learning for Instance Segmentation [9.62543698736491]
  We propose a generic method to obtain model-inherent uncertainty estimates within proposal-free instance segmentation models. We evaluate our method on the BBBC010 C. elegans dataset, where it yields competitive performance.
  arXiv  Detail & Related papers  (2020-08-24T19:51:48Z)
• On uncertainty estimation in active learning for image segmentation [7.05949591248206]
  Uncertainty estimation is important for interpreting the trustworthiness of machine learning models in many applications. In this paper, we explore uncertainty calibration within an active learning framework for medical image segmentation. We observe that selecting regions to annotate instead of full images leads to more well-calibrated models.
  arXiv  Detail & Related papers  (2020-07-13T13:20:32Z)
• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning [70.72363097550483]
  In this study, we focus on in-domain uncertainty for image classification. To provide more insight in this study, we introduce the deep ensemble equivalent score (DEE)
  arXiv  Detail & Related papers  (2020-02-15T23:28:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.