Conformal Prediction Intervals with Temporal Dependence
- URL: http://arxiv.org/abs/2205.12940v2
- Date: Thu, 26 May 2022 02:57:17 GMT
- Title: Conformal Prediction Intervals with Temporal Dependence
- Authors: Zhen Lin, Shubhendu Trivedi, Jimeng Sun
- Abstract summary: Cross-sectional prediction is common in many domains such as healthcare.
We focus on the task of constructing valid prediction intervals (PIs) in time-series regression with a cross-section.
- Score: 40.282423098764404
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Cross-sectional prediction is common in many domains such as healthcare,
including forecasting tasks using electronic health records, where different
patients form a cross-section. We focus on the task of constructing valid
prediction intervals (PIs) in time-series regression with a cross-section. A
prediction interval is considered valid if it covers the true response with (a
pre-specified) high probability. We first distinguish between two notions of
validity in such a setting: cross-sectional and longitudinal. Cross-sectional
validity is concerned with validity across the cross-section of the time series
data, while longitudinal validity accounts for the temporal dimension. Coverage
guarantees along both these dimensions are ideally desirable; however, we show
that distribution-free longitudinal validity is theoretically impossible.
Despite this limitation, we propose Conformal Prediction with Temporal
Dependence (CPTD), a procedure which is able to maintain strict cross-sectional
validity while improving longitudinal coverage. CPTD is post-hoc and
light-weight, and can easily be used in conjunction with any prediction model
as long as a calibration set is available. We focus on neural networks due to
their ability to model complicated data such as diagnosis codes for time-series
regression, and perform extensive experimental validation to verify the
efficacy of our approach. We find that CPTD outperforms baselines on a variety
of datasets by improving longitudinal coverage and often providing more
efficient (narrower) PIs.
Related papers
- Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
We develop a principled procedure for determining when a large language model should abstain from responding in a general domain.
We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate)
Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
arXiv Detail & Related papers (2024-04-04T11:32:03Z) - Conformal Predictions for Longitudinal Data [0.0]
We introduce Longitudinal Predictive Conformal Inference (L PCI), a distribution-free conformal prediction algorithm for longitudinal data.
Our experiments demonstrate that LPCI outperforms existing benchmarks in terms of longitudinal coverage rates.
The robust performance of LPCI in generating reliable prediction intervals for longitudinal data underscores its potential for broad applications.
arXiv Detail & Related papers (2023-10-04T14:51:07Z) - Conformal Prediction with Temporal Quantile Adjustments [40.282423098764404]
We develop a method to construct efficient and valid prediction intervals (PIs) for regression on cross-sectional time series data.
We validate TQA's performance through extensive experimentation.
arXiv Detail & Related papers (2022-05-20T03:31:03Z) - TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance.
We propose a versatile method that estimates joint distributions using an attention-based decoder.
We show that our model produces state-of-the-art predictions on several real-world datasets.
arXiv Detail & Related papers (2022-02-07T21:37:29Z) - Benefit-aware Early Prediction of Health Outcomes on Multivariate EEG
Time Series [37.15225732922409]
Motivated by this real-world application, we present BeneFitter that infuses the incurred savings from an early prediction as well as the cost from misclassification into a unified domain-specific target called benefit.
BeneFitter is efficient and fast, with training time linear in the number of input sequences, and can operate in real-time for decision-making.
arXiv Detail & Related papers (2021-11-11T02:54:36Z) - Sequential Diagnosis Prediction with Transformer and Ontological
Representation [35.88195694025553]
We propose an end-to-end robust transformer-based model called SETOR to handle irregular intervals between a patient's visits with admitted timestamps and length of stay in each visit.
Experiments conducted on two real-world healthcare datasets show that, our sequential diagnoses prediction model SETOR achieves better predictive results than previous state-of-the-art approaches.
arXiv Detail & Related papers (2021-09-07T13:09:55Z) - Interval-censored Hawkes processes [82.87738318505582]
We propose a model to estimate the parameters of a Hawkes process in interval-censored settings.
We show how a non-homogeneous approximation to the Hawkes admits a tractable likelihood in the interval-censored setting.
arXiv Detail & Related papers (2021-04-16T07:29:04Z) - Cross-validation: what does it estimate and how well does it do it? [2.049702429898688]
Cross-validation is a widely-used technique to estimate prediction error, but its behavior is complex and not fully understood.
We prove that this is not the case for the linear model fit by ordinary least squares; rather it estimates the average prediction error of models fit on other unseen training sets drawn from the same population.
arXiv Detail & Related papers (2021-04-01T17:58:54Z) - DeepRite: Deep Recurrent Inverse TreatmEnt Weighting for Adjusting
Time-varying Confounding in Modern Longitudinal Observational Data [68.29870617697532]
We propose Deep Recurrent Inverse TreatmEnt weighting (DeepRite) for time-varying confounding in longitudinal data.
DeepRite is shown to recover the ground truth from synthetic data, and estimate unbiased treatment effects from real data.
arXiv Detail & Related papers (2020-10-28T15:05:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.