Improving EO Foundation Models with Confidence Assessment for enhanced Semantic segmentation

• URL: http://arxiv.org/abs/2406.18279v2
• Date: Fri, 22 Nov 2024 08:45:55 GMT
• Title: Improving EO Foundation Models with Confidence Assessment for enhanced Semantic segmentation
• Authors: Nikolaos Dionelis, Nicolas Longepe,
• Abstract summary: We develop a Confidence Assessment for enhanced Semantic segmentation (CAS) model. It evaluates confidence at both the segment and pixel levels, providing both labels and confidence scores as output. This work has significant applications, particularly in evaluating EO Foundation Models on semantic segmentation downstream tasks.
• Score: 0.0
• License:
• Abstract: Confidence assessments of semantic segmentation algorithms are important. Ideally, deep learning models should have the ability to predict in advance whether their output is likely to be incorrect. Assessing the confidence levels of model predictions in Earth Observation (EO) classification is essential, as it can enhance semantic segmentation performance and help prevent further exploitation of the results in case of erroneous prediction. The model we developed, Confidence Assessment for enhanced Semantic segmentation (CAS), evaluates confidence at both the segment and pixel levels, providing both labels and confidence scores as output. Our model, CAS, identifies segments with incorrect predicted labels using the proposed combined confidence metric, refines the model, and enhances its performance. This work has significant applications, particularly in evaluating EO Foundation Models on semantic segmentation downstream tasks, such as land cover classification using Sentinel-2 satellite data. The evaluation results show that this strategy is effective and that the proposed model CAS outperforms other baseline models.

Related papers

• SASWISE-UE: Segmentation and Synthesis with Interpretable Scalable Ensembles for Uncertainty Estimation [6.082812294410541]
  This paper introduces an efficient sub-model ensemble framework aimed at enhancing the interpretability of medical deep learning models. By generating uncertainty maps, this framework enables end-users to evaluate the reliability of model outputs.
  arXiv  Detail & Related papers  (2024-11-08T04:37:55Z)
• Language Model Preference Evaluation with Multiple Weak Evaluators [78.53743237977677]
  GED (Preference Graph Ensemble and Denoise) is a novel approach that leverages multiple model-based evaluators to construct preference graphs. We show that GED outperforms baseline methods in model ranking, response selection, and model alignment tasks.
  arXiv  Detail & Related papers  (2024-10-14T01:57:25Z)
• Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models [69.38024658668887]
  Current evaluation method for event extraction relies on token-level exact match. We propose RAEE, an automatic evaluation framework that accurately assesses event extraction results at semantic-level instead of token-level.
  arXiv  Detail & Related papers  (2024-10-12T07:54:01Z)
• A Probabilistic Perspective on Unlearning and Alignment for Large Language Models [48.96686419141881]
  We introduce the first formal probabilistic evaluation framework in Large Language Models (LLMs) We derive novel metrics with high-probability guarantees concerning the output distribution of a model. Our metrics are application-independent and allow practitioners to make more reliable estimates about model capabilities before deployment.
  arXiv  Detail & Related papers  (2024-10-04T15:44:23Z)
• DECIDER: Leveraging Foundation Model Priors for Improved Model Failure Detection and Explanation [18.77296551727931]
  We propose DECIDER, a novel approach that leverages priors from large language models (LLMs) and vision-language models (VLMs) to detect failures in image models. DECIDER consistently achieves state-of-the-art failure detection performance, significantly outperforming baselines in terms of the overall Matthews correlation coefficient.
  arXiv  Detail & Related papers  (2024-08-01T07:08:11Z)
• Self-supervised co-salient object detection via feature correspondence at multiple scales [27.664016341526988]
  This paper introduces a novel two-stage self-supervised approach for detecting co-occurring salient objects (CoSOD) in image groups without requiring segmentation annotations. We train a self-supervised network that detects co-salient regions by computing local patch-level feature correspondences across images. In experiments on three CoSOD benchmark datasets, our model outperforms the corresponding state-of-the-art models by a huge margin.
  arXiv  Detail & Related papers  (2024-03-17T06:21:21Z)
• Towards Better Certified Segmentation via Diffusion Models [62.21617614504225]
  segmentation models can be vulnerable to adversarial perturbations, which hinders their use in critical-decision systems like healthcare or autonomous driving. Recently, randomized smoothing has been proposed to certify segmentation predictions by adding Gaussian noise to the input to obtain theoretical guarantees. In this paper, we address the problem of certifying segmentation prediction using a combination of randomized smoothing and diffusion models.
  arXiv  Detail & Related papers  (2023-06-16T16:30:39Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)
• Estimating Model Performance under Domain Shifts with Class-Specific Confidence Scores [25.162667593654206]
  We introduce class-wise calibration within the framework of performance estimation for imbalanced datasets. We conduct experiments on four tasks and find the proposed modifications consistently improve the estimation accuracy for imbalanced datasets.
  arXiv  Detail & Related papers  (2022-07-20T15:04:32Z)
• Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencoder [23.860842627883187]
  We teach the model to capture broader variations of the feature distributions with a novel noise-enhanced supervised autoencoder (NSAE) NSAE trains the model by jointly reconstructing inputs and predicting the labels of inputs as well as their reconstructed pairs. We also take advantage of NSAE structure and propose a two-step fine-tuning procedure that achieves better adaption and improves classification performance in the target domain.
  arXiv  Detail & Related papers  (2021-08-11T04:45:56Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.