SURE-Val: Safe Urban Relevance Extension and Validation
- URL: http://arxiv.org/abs/2308.02266v1
- Date: Fri, 4 Aug 2023 11:44:08 GMT
- Title: SURE-Val: Safe Urban Relevance Extension and Validation
- Authors: Kai Storms, Ken Mori, Steven Peters
- Abstract summary: This work adopts an existing method to define relevance in the highway domain and expands it to the urban domain.
While different conceptualizations and definitions of relevance are present in literature, there is a lack of methods to validate these definitions.
The validation leverages the idea that removing irrelevant objects should not influence a prediction component which reflects human driving behavior.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: To evaluate perception components of an automated driving system, it is
necessary to define the relevant objects. While the urban domain is popular
among perception datasets, relevance is insufficiently specified for this
domain. Therefore, this work adopts an existing method to define relevance in
the highway domain and expands it to the urban domain. While different
conceptualizations and definitions of relevance are present in literature,
there is a lack of methods to validate these definitions. Therefore, this work
presents a novel relevance validation method leveraging a motion prediction
component. The validation leverages the idea that removing irrelevant objects
should not influence a prediction component which reflects human driving
behavior. The influence on the prediction is quantified by considering the
statistical distribution of prediction performance across a large-scale
dataset. The validation procedure is verified using criteria specifically
designed to exclude relevant objects. The validation method is successfully
applied to the relevance criteria from this work, thus supporting their
validity.
Related papers
- Conservative Estimation of Perception Relevance of Dynamic Objects for
Safe Trajectories in Automotive Scenarios [0.0]
The concept of relevance currently remains insufficiently defined and specified.
We propose a novel methodology to overcome this challenge by exemplary application to collision safety in the highway domain.
We present a conservative estimation which dynamic objects are relevant for perception and need to be considered for a complete evaluation.
arXiv Detail & Related papers (2023-07-20T13:43:48Z) - Uncertainty Calibration and its Application to Object Detection [0.0]
In this work, we examine the semantic uncertainty (which object type?) as well as the spatial uncertainty.
We evaluate if the predicted uncertainties of an object detection model match with the observed error that is achieved on real-world data.
arXiv Detail & Related papers (2023-02-06T08:41:07Z) - Exploring validation metrics for offline model-based optimisation with
diffusion models [50.404829846182764]
In model-based optimisation (MBO) we are interested in using machine learning to design candidates that maximise some measure of reward with respect to a black box function called the (ground truth) oracle.
While an approximation to the ground oracle can be trained and used in place of it during model validation to measure the mean reward over generated candidates, the evaluation is approximate and vulnerable to adversarial examples.
This is encapsulated under our proposed evaluation framework which is also designed to measure extrapolation.
arXiv Detail & Related papers (2022-11-19T16:57:37Z) - Measuring Fairness of Text Classifiers via Prediction Sensitivity [63.56554964580627]
ACCUMULATED PREDICTION SENSITIVITY measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features.
We show that the metric can be theoretically linked with a specific notion of group fairness (statistical parity) and individual fairness.
arXiv Detail & Related papers (2022-03-16T15:00:33Z) - Off-Policy Confidence Interval Estimation with Confounded Markov
Decision Process [14.828039846764549]
We show that with some auxiliary variables that mediate the effect of actions on the system dynamics, the target policy's value is identifiable in a confounded Markov decision process.
Our method is justified by theoretical results, simulated and real datasets obtained from ridesharing companies.
arXiv Detail & Related papers (2022-02-22T00:03:48Z) - Trajectory Forecasting from Detection with Uncertainty-Aware Motion
Encoding [121.66374635092097]
Trajectories obtained from object detection and tracking are inevitably noisy.
We propose a trajectory predictor directly based on detection results without relying on explicitly formed trajectories.
arXiv Detail & Related papers (2022-02-03T09:09:56Z) - Uncertainty Aware Proposal Segmentation for Unknown Object Detection [13.249453757295083]
This paper proposes to exploit additional predictions of semantic segmentation models and quantifying its confidences.
We use object proposals generated by Region Proposal Network (RPN) and adapt distance aware uncertainty estimation of semantic segmentation.
The augmented object proposals are then used to train a classifier for known vs. unknown objects categories.
arXiv Detail & Related papers (2021-11-25T01:53:05Z) - CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings.
In this work, we propose a novel sampling-free uncertainty estimation method for object detection.
We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
arXiv Detail & Related papers (2021-10-04T17:59:31Z) - Learning Uncertainty For Safety-Oriented Semantic Segmentation In
Autonomous Driving [77.39239190539871]
We show how uncertainty estimation can be leveraged to enable safety critical image segmentation in autonomous driving.
We introduce a new uncertainty measure based on disagreeing predictions as measured by a dissimilarity function.
We show experimentally that our proposed approach is much less computationally intensive at inference time than competing methods.
arXiv Detail & Related papers (2021-05-28T09:23:05Z) - Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain
Adaptive Semantic Segmentation [49.295165476818866]
This paper focuses on the unsupervised domain adaptation of transferring the knowledge from the source domain to the target domain in the context of semantic segmentation.
Existing approaches usually regard the pseudo label as the ground truth to fully exploit the unlabeled target-domain data.
This paper proposes to explicitly estimate the prediction uncertainty during training to rectify the pseudo label learning.
arXiv Detail & Related papers (2020-03-08T12:37:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.