Uncertainty-aware Panoptic Segmentation
- URL: http://arxiv.org/abs/2206.14554v1
- Date: Wed, 29 Jun 2022 12:07:21 GMT
- Title: Uncertainty-aware Panoptic Segmentation
- Authors: Kshitij Sirohi, Sajad Marvi, Daniel B\"uscher, Wolfram Burgard
- Abstract summary: We introduce the novel task of uncertainty-aware panoptic segmentation.
It aims to predict per-pixel semantic and instance segmentations, together with per-pixel uncertainty estimates.
We propose the novel top-down Evidential Panoptic Network (EvPSNet) to solve this task.
- Score: 21.89063036529791
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reliable scene understanding is indispensable for modern autonomous systems.
Current learning-based methods typically try to maximize their performance
based on segmentation metrics that only consider the quality of the
segmentation. However, for the safe operation of a system in the real world it
is crucial to consider the uncertainty in the prediction as well. In this work,
we introduce the novel task of uncertainty-aware panoptic segmentation, which
aims to predict per-pixel semantic and instance segmentations, together with
per-pixel uncertainty estimates. We define two novel metrics to facilitate its
quantitative analysis, the uncertainty-aware Panoptic Quality (uPQ) and the
panoptic Expected Calibration Error (pECE). We further propose the novel
top-down Evidential Panoptic Segmentation Network (EvPSNet) to solve this task.
Our architecture employs a simple yet effective probabilistic fusion module
that leverages the predicted uncertainties. Additionally, we propose a new
Lov\'asz evidential loss function to optimize the IoU for the segmentation
utilizing the probabilities provided by deep evidential learning. Furthermore,
we provide several strong baselines combining state-of-the-art panoptic
segmentation networks with sampling-free uncertainty estimation techniques.
Extensive evaluations show that our EvPSNet achieves the new state-of-the-art
for the standard Panoptic Quality (PQ), as well as for our uncertainty-aware
panoptic metrics.
Related papers
- Know Where You're Uncertain When Planning with Multimodal Foundation Models: A Formal Framework [54.40508478482667]
We present a comprehensive framework to disentangle, quantify, and mitigate uncertainty in perception and plan generation.
We propose methods tailored to the unique properties of perception and decision-making.
We show that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines.
arXiv Detail & Related papers (2024-11-03T17:32:00Z) - Uncertainty Quantification for Bird's Eye View Semantic Segmentation: Methods and Benchmarks [10.193504550494486]
This paper introduces a benchmark for predictive uncertainty quantification in BEV segmentation.
It focuses on the effectiveness of predicted uncertainty in identifying misclassified and out-of-distribution pixels, as well as calibration.
We propose the Uncertainty-Focal-Cross-Entropy loss, designed for highly imbalanced data, which consistently improves the segmentation quality and calibration.
arXiv Detail & Related papers (2024-05-31T16:32:46Z) - Segmentation Re-thinking Uncertainty Estimation Metrics for Semantic Segmentation [12.532289778772185]
semantic segmentation is a fundamental application within machine learning.
The metric known as PAvPU (Patch Accuracy versus Patch Uncertainty) has been developed as a specialized tool for evaluating entropy-based uncertainty in image segmentation tasks.
Our investigation identifies three core deficiencies within the PAvPU framework and proposes robust solutions.
arXiv Detail & Related papers (2024-03-28T20:34:02Z) - ProPanDL: A Modular Architecture for Uncertainty-Aware Panoptic
Segmentation [0.0]
We introduce ProPanDL, a family of networks capable of uncertainty-aware panoptic segmentation.
We implement and evaluate ProPanDL variants capable of estimating both parametric (Variance Network) and parameter-free (SampleNet) distributions.
Our results demonstrate that ProPanDL is capable of estimating well-calibrated and meaningful output distributions while still retaining strong performance on the base panoptic segmentation task.
arXiv Detail & Related papers (2023-04-17T22:31:23Z) - Uncertainty Estimation by Fisher Information-based Evidential Deep
Learning [61.94125052118442]
Uncertainty estimation is a key factor that makes deep learning reliable in practical applications.
We propose a novel method, Fisher Information-based Evidential Deep Learning ($mathcalI$-EDL)
In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes.
arXiv Detail & Related papers (2023-03-03T16:12:59Z) - Uncertainty-aware LiDAR Panoptic Segmentation [21.89063036529791]
We introduce a novel approach for solving the task of uncertainty-aware panoptic segmentation using LiDAR point clouds.
Our proposed EvLPSNet network is the first to solve this task efficiently in a sampling-free manner.
We provide several strong baselines combining state-of-the-art panoptic segmentation networks with sampling-free uncertainty estimation techniques.
arXiv Detail & Related papers (2022-10-10T07:54:57Z) - NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural
Networks [151.03112356092575]
We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution.
We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
arXiv Detail & Related papers (2022-02-07T12:30:45Z) - CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings.
In this work, we propose a novel sampling-free uncertainty estimation method for object detection.
We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
arXiv Detail & Related papers (2021-10-04T17:59:31Z) - Learning Uncertainty For Safety-Oriented Semantic Segmentation In
Autonomous Driving [77.39239190539871]
We show how uncertainty estimation can be leveraged to enable safety critical image segmentation in autonomous driving.
We introduce a new uncertainty measure based on disagreeing predictions as measured by a dissimilarity function.
We show experimentally that our proposed approach is much less computationally intensive at inference time than competing methods.
arXiv Detail & Related papers (2021-05-28T09:23:05Z) - The Benefit of the Doubt: Uncertainty Aware Sensing for Edge Computing
Platforms [10.86298377998459]
We propose an efficient framework for predictive uncertainty estimation in NNs deployed on embedded edge systems.
The framework is built from the ground up to provide predictive uncertainty based only on one forward pass.
Our approach not only obtains robust and accurate uncertainty estimations but also outperforms state-of-the-art methods in terms of systems performance.
arXiv Detail & Related papers (2021-02-11T11:44:32Z) - Approaching Neural Network Uncertainty Realism [53.308409014122816]
Quantifying or at least upper-bounding uncertainties is vital for safety-critical systems such as autonomous vehicles.
We evaluate uncertainty realism -- a strict quality criterion -- with a Mahalanobis distance-based statistical test.
We adopt it to the automotive domain and show that it significantly improves uncertainty realism compared to a plain encoder-decoder model.
arXiv Detail & Related papers (2021-01-08T11:56:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.