Sequential Predictive Conformal Inference for Time Series
- URL: http://arxiv.org/abs/2212.03463v3
- Date: Tue, 30 May 2023 04:02:26 GMT
- Title: Sequential Predictive Conformal Inference for Time Series
- Authors: Chen Xu, Yao Xie
- Abstract summary: We present a new distribution-free conformal prediction algorithm for sequential data (e.g., time series)
We specifically account for the nature that time series data are non-exchangeable, and thus many existing conformal prediction algorithms are not applicable.
- Score: 16.38369532102931
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present a new distribution-free conformal prediction algorithm for
sequential data (e.g., time series), called the \textit{sequential predictive
conformal inference} (\texttt{SPCI}). We specifically account for the nature
that time series data are non-exchangeable, and thus many existing conformal
prediction algorithms are not applicable. The main idea is to adaptively
re-estimate the conditional quantile of non-conformity scores (e.g., prediction
residuals), upon exploiting the temporal dependence among them. More precisely,
we cast the problem of conformal prediction interval as predicting the quantile
of a future residual, given a user-specified point prediction algorithm.
Theoretically, we establish asymptotic valid conditional coverage upon
extending consistency analyses in quantile regression. Using simulation and
real-data experiments, we demonstrate a significant reduction in interval width
of \texttt{SPCI} compared to other existing methods under the desired empirical
coverage.
Related papers
- Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution.
Our method consistently outperforms existing approaches in terms of conditional coverage.
arXiv Detail & Related papers (2024-07-01T20:44:48Z) - Selective Nonparametric Regression via Testing [54.20569354303575]
We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point.
Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
arXiv Detail & Related papers (2023-09-28T13:04:11Z) - Conformal Prediction with Missing Values [19.18178194789968]
We first show that the marginal coverage guarantee of conformal prediction holds on imputed data for any missingness distribution.
We then show that a universally consistent quantile regression algorithm trained on the imputed data is Bayes optimal for the pinball risk.
arXiv Detail & Related papers (2023-06-05T09:28:03Z) - Nonparametric Quantile Regression: Non-Crossing Constraints and
Conformal Prediction [2.654399717608053]
We propose a nonparametric quantile regression method using deep neural networks with a rectified linear unit penalty function to avoid quantile crossing.
We establish non-asymptotic upper bounds for the excess risk of the proposed nonparametric quantile regression function estimators.
Numerical experiments including simulation studies and a real data example are conducted to demonstrate the effectiveness of the proposed method.
arXiv Detail & Related papers (2022-10-18T20:59:48Z) - Predictive Inference with Feature Conformal Prediction [80.77443423828315]
We propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces.
From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions.
Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods.
arXiv Detail & Related papers (2022-10-01T02:57:37Z) - Probabilistic Conformal Prediction Using Conditional Random Samples [73.26753677005331]
PCP is a predictive inference algorithm that estimates a target variable by a discontinuous predictive set.
It is efficient and compatible with either explicit or implicit conditional generative models.
arXiv Detail & Related papers (2022-06-14T03:58:03Z) - Efficient and Differentiable Conformal Prediction with General Function
Classes [96.74055810115456]
We propose a generalization of conformal prediction to multiple learnable parameters.
We show that it achieves approximate valid population coverage and near-optimal efficiency within class.
Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly.
arXiv Detail & Related papers (2022-02-22T18:37:23Z) - Comparing Sequential Forecasters [35.38264087676121]
Consider two forecasters, each making a single prediction for a sequence of events over time.
How might we compare these forecasters, either online or post-hoc, while avoiding unverifiable assumptions on how the forecasts and outcomes were generated?
We present novel sequential inference procedures for estimating the time-varying difference in forecast scores.
We empirically validate our approaches by comparing real-world baseball and weather forecasters.
arXiv Detail & Related papers (2021-09-30T22:54:46Z) - Conformal histogram regression [15.153110906331737]
This paper develops a conformal method to compute prediction intervals for non-parametric regression that can automatically adapt to skewed data.
Leveraging black-box machine learning algorithms, it translates their output into the shortest prediction intervals with approximate conditional coverage.
The resulting prediction intervals provably have marginal coverage in finite samples, while achieving conditional coverage and optimal length if the black-box model is consistent.
arXiv Detail & Related papers (2021-05-18T18:05:02Z) - Conformal prediction for time series [16.38369532102931]
textttEnbPI wraps around ensemble predictors, which is closely related to conformal prediction (CP) but does not require data exchangeability.
We perform extensive simulation and real-data analyses to demonstrate its effectiveness compared with existing methods.
arXiv Detail & Related papers (2020-10-18T21:05:32Z) - Ambiguity in Sequential Data: Predicting Uncertain Futures with
Recurrent Models [110.82452096672182]
We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data.
We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
arXiv Detail & Related papers (2020-03-10T09:15:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.