Conformal Prediction Intervals with Temporal Dependence
- URL: http://arxiv.org/abs/2205.12940v2
- Date: Thu, 26 May 2022 02:57:17 GMT
- Title: Conformal Prediction Intervals with Temporal Dependence
- Authors: Zhen Lin, Shubhendu Trivedi, Jimeng Sun
- Abstract summary: Cross-sectional prediction is common in many domains such as healthcare.
We focus on the task of constructing valid prediction intervals (PIs) in time-series regression with a cross-section.
- Score: 40.282423098764404
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Cross-sectional prediction is common in many domains such as healthcare,
including forecasting tasks using electronic health records, where different
patients form a cross-section. We focus on the task of constructing valid
prediction intervals (PIs) in time-series regression with a cross-section. A
prediction interval is considered valid if it covers the true response with (a
pre-specified) high probability. We first distinguish between two notions of
validity in such a setting: cross-sectional and longitudinal. Cross-sectional
validity is concerned with validity across the cross-section of the time series
data, while longitudinal validity accounts for the temporal dimension. Coverage
guarantees along both these dimensions are ideally desirable; however, we show
that distribution-free longitudinal validity is theoretically impossible.
Despite this limitation, we propose Conformal Prediction with Temporal
Dependence (CPTD), a procedure which is able to maintain strict cross-sectional
validity while improving longitudinal coverage. CPTD is post-hoc and
light-weight, and can easily be used in conjunction with any prediction model
as long as a calibration set is available. We focus on neural networks due to
their ability to model complicated data such as diagnosis codes for time-series
regression, and perform extensive experimental validation to verify the
efficacy of our approach. We find that CPTD outperforms baselines on a variety
of datasets by improving longitudinal coverage and often providing more
efficient (narrower) PIs.
Related papers
- Conformal Inference of Individual Treatment Effects Using Conditional Density Estimates [3.7307776333361122]
Current state-of-the-art approaches, while providing valid prediction intervals, often yield overly conservative prediction intervals.
In this work, we introduce a conformal inference approach to ITE using the conditional density of the outcome.
We show that our prediction intervals are not only marginally valid but are narrower than existing methods.
arXiv Detail & Related papers (2025-01-24T21:46:37Z) - Stabilized Neural Prediction of Potential Outcomes in Continuous Time [23.128421664169654]
We propose a new method called stabilized continuous time inverse propensity network (SCIP-Net)
It is the first neural method that performs proper adjustments for time-varying confounding in continuous time.
arXiv Detail & Related papers (2024-10-04T15:29:11Z) - Enhancing reliability in prediction intervals using point forecasters: Heteroscedastic Quantile Regression and Width-Adaptive Conformal Inference [0.0]
We argue that standard measures alone are inadequate when constructing prediction intervals.
We propose combining Heteroscedastic Quantile Regression with Width-Adaptive Conformal Inference.
Our results show that this combined approach meets or surpasses typical benchmarks for validity and efficiency.
arXiv Detail & Related papers (2024-06-21T06:51:13Z) - Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
We develop a principled procedure for determining when a large language model should abstain from responding in a general domain.
We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate)
Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
arXiv Detail & Related papers (2024-04-04T11:32:03Z) - Conformal Predictions for Longitudinal Data [0.0]
We introduce Longitudinal Predictive Conformal Inference (L PCI), a distribution-free conformal prediction algorithm for longitudinal data.
Our experiments demonstrate that LPCI outperforms existing benchmarks in terms of longitudinal coverage rates.
The robust performance of LPCI in generating reliable prediction intervals for longitudinal data underscores its potential for broad applications.
arXiv Detail & Related papers (2023-10-04T14:51:07Z) - Conformal Prediction with Temporal Quantile Adjustments [40.282423098764404]
We develop a method to construct efficient and valid prediction intervals (PIs) for regression on cross-sectional time series data.
We validate TQA's performance through extensive experimentation.
arXiv Detail & Related papers (2022-05-20T03:31:03Z) - TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance.
We propose a versatile method that estimates joint distributions using an attention-based decoder.
We show that our model produces state-of-the-art predictions on several real-world datasets.
arXiv Detail & Related papers (2022-02-07T21:37:29Z) - Interval-censored Hawkes processes [82.87738318505582]
We propose a model to estimate the parameters of a Hawkes process in interval-censored settings.
We show how a non-homogeneous approximation to the Hawkes admits a tractable likelihood in the interval-censored setting.
arXiv Detail & Related papers (2021-04-16T07:29:04Z) - DeepRite: Deep Recurrent Inverse TreatmEnt Weighting for Adjusting
Time-varying Confounding in Modern Longitudinal Observational Data [68.29870617697532]
We propose Deep Recurrent Inverse TreatmEnt weighting (DeepRite) for time-varying confounding in longitudinal data.
DeepRite is shown to recover the ground truth from synthetic data, and estimate unbiased treatment effects from real data.
arXiv Detail & Related papers (2020-10-28T15:05:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.