Estimating Uncertainty in Landslide Segmentation Models

• URL: http://arxiv.org/abs/2311.11138v2
• Date: Mon, 25 Mar 2024 16:10:20 GMT
• Title: Estimating Uncertainty in Landslide Segmentation Models
• Authors: Savinay Nagendra, Chaopeng Shen, Daniel Kifer,
• Abstract summary: Landslides are a recurring, widespread hazard. Preparation and mitigation efforts can be aided by a high-quality, large-scale dataset that covers global at-risk areas. Recent automated efforts focus on deep learning models for landslide segmentation from satellite imagery. Accurate and robust uncertainty estimates can enable low-cost oversight of auto-generated landslide databases to resolve errors, identify hard negative examples, and increase the size of labeled training data.
• Score: 7.537865319452023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Landslides are a recurring, widespread hazard. Preparation and mitigation efforts can be aided by a high-quality, large-scale dataset that covers global at-risk areas. Such a dataset currently does not exist and is impossible to construct manually. Recent automated efforts focus on deep learning models for landslide segmentation (pixel labeling) from satellite imagery. However, it is also important to characterize the uncertainty or confidence levels of such segmentations. Accurate and robust uncertainty estimates can enable low-cost (in terms of manual labor) oversight of auto-generated landslide databases to resolve errors, identify hard negative examples, and increase the size of labeled training data. In this paper, we evaluate several methods for assessing pixel-level uncertainty of the segmentation. Three methods that do not require architectural changes were compared, including Pre-Threshold activations, Monte-Carlo Dropout and Test-Time Augmentation -- a method that measures the robustness of predictions in the face of data augmentation. Experimentally, the quality of the latter method was consistently higher than the others across a variety of models and metrics in our dataset.

Related papers

• Masked Gamma-SSL: Learning Uncertainty Estimation via Masked Image Modeling [19.000718685399935]
  This work proposes a semantic segmentation network that produces high-quality uncertainty estimates in a single forward pass. We exploit general representations from foundation models and unlabelled datasets through a Masked Image Modeling (MIM) approach. For neural networks used in safety-critical applications, bias in the training data can lead to errors.
  arXiv  Detail & Related papers  (2024-02-27T15:49:54Z)
• Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise [62.997667081978825]
  In high-risk environments, deep learning models need to be able to judge their uncertainty and reject inputs when there is a significant chance of misclassification. We conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise.
  arXiv  Detail & Related papers  (2023-01-03T11:34:36Z)
• Uncertainty Reduction for 3D Point Cloud Self-Supervised Traversability Estimation [17.193700394066266]
  Self-supervised traversability estimation suffers from congenital uncertainties that appear according to the scarcity of negative information. We introduce a method to incorporate unlabeled data in order to leverage the uncertainty. We evaluate our approach on our own dataset, Dtrail', which is composed of a wide variety of negative data.
  arXiv  Detail & Related papers  (2022-11-21T06:24:41Z)
• Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity [7.3372471678239215]
  We assess the quality of estimates from a wide array of approaches and their dependence on the amount of available data. We find that while approaches based on pre-trained models and ensembles achieve the best results overall, the quality of uncertainty estimates can surprisingly suffer with more data.
  arXiv  Detail & Related papers  (2022-10-20T15:42:02Z)
• BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks [50.15201777970128]
  We propose BayesCap that learns a Bayesian identity mapping for the frozen model, allowing uncertainty estimation. BayesCap is a memory-efficient method that can be trained on a small fraction of the original dataset. We show the efficacy of our method on a wide variety of tasks with a diverse set of architectures.
  arXiv  Detail & Related papers  (2022-07-14T12:50:09Z)
• Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey [1.5469452301122175]
  Deep learning models rely on the abundance of labelled observations to train a prospective model. It is expensive to gather labelled observations of data, making the usage of deep learning models not ideal. In many situations different unlabelled data sources might be available. This raises the risk of a significant distribution mismatch between the labelled and unlabelled datasets.
  arXiv  Detail & Related papers  (2022-03-01T02:46:00Z)
• Improving Generalization via Uncertainty Driven Perturbations [107.45752065285821]
  We consider uncertainty-driven perturbations of the training data points. Unlike loss-driven perturbations, uncertainty-guided perturbations do not cross the decision boundary. We show that UDP is guaranteed to achieve the robustness margin decision on linear models.
  arXiv  Detail & Related papers  (2022-02-11T16:22:08Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Approaching Neural Network Uncertainty Realism [53.308409014122816]
  Quantifying or at least upper-bounding uncertainties is vital for safety-critical systems such as autonomous vehicles. We evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test. We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.
  arXiv  Detail & Related papers  (2021-01-08T11:56:12Z)
• Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures [13.890139530120164]
  We provide the first large scale evaluation of the empirical frequentist coverage properties of uncertainty quantification techniques. We find that, in general, some methods do achieve desirable coverage properties on in distribution samples, but that coverage is not maintained on out-of-distribution data.
  arXiv  Detail & Related papers  (2020-10-06T21:22:46Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.