Regression Trees for Fast and Adaptive Prediction Intervals
- URL: http://arxiv.org/abs/2402.07357v2
- Date: Tue, 13 Feb 2024 13:46:07 GMT
- Title: Regression Trees for Fast and Adaptive Prediction Intervals
- Authors: Luben M. C. Cabezas, Mateus P. Otto, Rafael Izbicki, Rafael B. Stern
- Abstract summary: We present a family of methods to calibrate prediction intervals for regression problems with local coverage guarantees.
We create a partition by training regression trees and Random Forests on conformity scores.
Our proposal is versatile, as it applies to various conformity scores and prediction settings.
- Score: 2.6763498831034043
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Predictive models make mistakes. Hence, there is a need to quantify the
uncertainty associated with their predictions. Conformal inference has emerged
as a powerful tool to create statistically valid prediction regions around
point predictions, but its naive application to regression problems yields
non-adaptive regions. New conformal scores, often relying upon quantile
regressors or conditional density estimators, aim to address this limitation.
Although they are useful for creating prediction bands, these scores are
detached from the original goal of quantifying the uncertainty around an
arbitrary predictive model. This paper presents a new, model-agnostic family of
methods to calibrate prediction intervals for regression problems with local
coverage guarantees. Our approach is based on pursuing the coarsest partition
of the feature space that approximates conditional coverage. We create this
partition by training regression trees and Random Forests on conformity scores.
Our proposal is versatile, as it applies to various conformity scores and
prediction settings and demonstrates superior scalability and performance
compared to established baselines in simulated and real-world datasets. We
provide a Python package clover that implements our methods using the standard
scikit-learn interface.
Related papers
- missForestPredict -- Missing data imputation for prediction settings [2.8461446020965435]
missForestPredict is a fast and user-friendly adaptation of the missForest imputation algorithm.
missForestPredict offers extended error monitoring and control over variables used in the imputation.
missForestPredict provides competitive results in prediction settings within short computation times.
arXiv Detail & Related papers (2024-07-02T17:45:46Z) - Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs.
We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint.
We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
arXiv Detail & Related papers (2024-06-05T13:36:38Z) - Domain-adaptive and Subgroup-specific Cascaded Temperature Regression
for Out-of-distribution Calibration [16.930766717110053]
We propose a novel meta-set-based cascaded temperature regression method for post-hoc calibration.
We partition each meta-set into subgroups based on predicted category and confidence level, capturing diverse uncertainties.
A regression network is then trained to derive category-specific and confidence-level-specific scaling, achieving calibration across meta-sets.
arXiv Detail & Related papers (2024-02-14T14:35:57Z) - Conformal Prediction with Missing Values [19.18178194789968]
We first show that the marginal coverage guarantee of conformal prediction holds on imputed data for any missingness distribution.
We then show that a universally consistent quantile regression algorithm trained on the imputed data is Bayes optimal for the pinball risk.
arXiv Detail & Related papers (2023-06-05T09:28:03Z) - Variational Inference with Coverage Guarantees in Simulation-Based Inference [18.818573945984873]
We propose Conformalized Amortized Neural Variational Inference (CANVI)
CANVI constructs conformalized predictors based on each candidate, compares the predictors using a metric known as predictive efficiency, and returns the most efficient predictor.
We prove lower bounds on the predictive efficiency of the regions produced by CANVI and explore how the quality of a posterior approximation relates to the predictive efficiency of prediction regions based on that approximation.
arXiv Detail & Related papers (2023-05-23T17:24:04Z) - Conformal prediction for the design problem [72.14982816083297]
In many real-world deployments of machine learning, we use a prediction algorithm to choose what data to test next.
In such settings, there is a distinct type of distribution shift between the training and test data.
We introduce a method to quantify predictive uncertainty in such settings.
arXiv Detail & Related papers (2022-02-08T02:59:12Z) - Private Prediction Sets [72.75711776601973]
Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy.
We present a framework that treats these two desiderata jointly.
We evaluate the method on large-scale computer vision datasets.
arXiv Detail & Related papers (2021-02-11T18:59:11Z) - Unlabelled Data Improves Bayesian Uncertainty Calibration under
Covariate Shift [100.52588638477862]
We develop an approximate Bayesian inference scheme based on posterior regularisation.
We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
arXiv Detail & Related papers (2020-06-26T13:50:19Z) - Individual Calibration with Randomized Forecasting [116.2086707626651]
We show that calibration for individual samples is possible in the regression setup if the predictions are randomized.
We design a training objective to enforce individual calibration and use it to train randomized regression functions.
arXiv Detail & Related papers (2020-06-18T05:53:10Z) - Censored Quantile Regression Forest [81.9098291337097]
We develop a new estimating equation that adapts to censoring and leads to quantile score whenever the data do not exhibit censoring.
The proposed procedure named it censored quantile regression forest, allows us to estimate quantiles of time-to-event without any parametric modeling assumption.
arXiv Detail & Related papers (2020-01-08T23:20:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.