Interpretable Machines: Constructing Valid Prediction Intervals with Random Forests

• URL: http://arxiv.org/abs/2103.05766v1
• Date: Tue, 9 Mar 2021 23:05:55 GMT
• Title: Interpretable Machines: Constructing Valid Prediction Intervals with Random Forests
• Authors: Burim Ramosaj
• Abstract summary: An important issue when using Machine Learning algorithms in recent research is the lack of interpretability. A contribution to this gap for the Random Forest Regression Learner is presented here. Several parametric and non-parametric prediction intervals are provided for Random Forest point predictions. A thorough investigation through Monte-Carlo simulation is conducted evaluating the performance of the proposed methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: An important issue when using Machine Learning algorithms in recent research is the lack of interpretability. Although these algorithms provide accurate point predictions for various learning problems, uncertainty estimates connected with point predictions are rather sparse. A contribution to this gap for the Random Forest Regression Learner is presented here. Based on its Out-of-Bag procedure, several parametric and non-parametric prediction intervals are provided for Random Forest point predictions and theoretical guarantees for its correct coverage probability is delivered. In a second part, a thorough investigation through Monte-Carlo simulation is conducted evaluating the performance of the proposed methods from three aspects: (i) Analyzing the correct coverage rate of the proposed prediction intervals, (ii) Inspecting interval width and (iii) Verifying the competitiveness of the proposed intervals with existing methods. The simulation yields that the proposed prediction intervals are robust towards non-normal residual distributions and are competitive by providing correct coverage rates and comparably narrow interval lengths, even for comparably small samples.

Related papers

• Conformalized Interval Arithmetic with Symmetric Calibration [9.559062601251464]
  We develop conformal prediction intervals for single target to the prediction interval for sum of multiple targets. We show that our method outperforms existing conformalized approaches as well as non-conformal approaches.
  arXiv  Detail & Related papers  (2024-08-20T15:27:18Z)
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
  Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs. We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint. We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
  arXiv  Detail & Related papers  (2024-06-05T13:36:38Z)
• Uncertainty Quantification via Stable Distribution Propagation [60.065272548502]
  We propose a new approach for propagating stable probability distributions through neural networks. Our method is based on local linearization, which we show to be an optimal approximation in terms of total variation distance for the ReLU non-linearity.
  arXiv  Detail & Related papers  (2024-02-13T09:40:19Z)
• Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification [59.81904428056924]
  We introduce SMASH: a Score MAtching estimator for learning markedPs with uncertainty quantification. Specifically, our framework adopts a normalization-free objective by estimating the pseudolikelihood of markedPs through score-matching. The superior performance of our proposed framework is demonstrated through extensive experiments in both event prediction and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-10-25T02:37:51Z)
• Prediction-Powered Inference [68.97619568620709]
  Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning.
  arXiv  Detail & Related papers  (2023-01-23T18:59:28Z)
• CovarianceNet: Conditional Generative Model for Correct Covariance Prediction in Human Motion Prediction [71.31516599226606]
  We present a new method to correctly predict the uncertainty associated with the predicted distribution of future trajectories. Our approach, CovariaceNet, is based on a Conditional Generative Model with Gaussian latent variables.
  arXiv  Detail & Related papers  (2021-09-07T09:38:24Z)
• RFpredInterval: An R Package for Prediction Intervals with Random Forests and Boosted Forests [0.0]
  We have developed a comprehensive R package, RFpredInterval, that integrates 16 methods to build prediction intervals with random forests and boosted forests. The methods implemented in the package are a new method to build prediction intervals with boosted forests (PIBF) and 15 different variants to produce prediction intervals with random forests proposed by Roy and Larocque ( 2020) The results show that the proposed method is very competitive and, globally, it outperforms the competing methods.
  arXiv  Detail & Related papers  (2021-06-15T15:27:50Z)
• Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
  We describe two types of forecasting problems: regular grid-based and graph-based. We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
  arXiv  Detail & Related papers  (2021-05-25T14:35:46Z)
• Conformal prediction for time series [16.38369532102931]
  textttEnbPI wraps around ensemble predictors, which is closely related to conformal prediction (CP) but does not require data exchangeability. We perform extensive simulation and real-data analyses to demonstrate its effectiveness compared with existing methods.
  arXiv  Detail & Related papers  (2020-10-18T21:05:32Z)
• Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles [4.521131595149397]
  We present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.
  arXiv  Detail & Related papers  (2020-07-19T13:46:34Z)
• Conformal Prediction Intervals for Neural Networks Using Cross Validation [0.0]
  Neural networks are among the most powerful nonlinear models used to address supervised learning problems. We propose the $k$-fold prediction interval method to construct prediction intervals for neural networks based on $k$-fold cross validation.
  arXiv  Detail & Related papers  (2020-06-30T16:23:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.