Conformal Prediction Intervals with Temporal Dependence
- URL: http://arxiv.org/abs/2205.12940v2
- Date: Thu, 26 May 2022 02:57:17 GMT
- Title: Conformal Prediction Intervals with Temporal Dependence
- Authors: Zhen Lin, Shubhendu Trivedi, Jimeng Sun
- Abstract summary: Cross-sectional prediction is common in many domains such as healthcare.
We focus on the task of constructing valid prediction intervals (PIs) in time-series regression with a cross-section.
- Score: 40.282423098764404
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Cross-sectional prediction is common in many domains such as healthcare,
including forecasting tasks using electronic health records, where different
patients form a cross-section. We focus on the task of constructing valid
prediction intervals (PIs) in time-series regression with a cross-section. A
prediction interval is considered valid if it covers the true response with (a
pre-specified) high probability. We first distinguish between two notions of
validity in such a setting: cross-sectional and longitudinal. Cross-sectional
validity is concerned with validity across the cross-section of the time series
data, while longitudinal validity accounts for the temporal dimension. Coverage
guarantees along both these dimensions are ideally desirable; however, we show
that distribution-free longitudinal validity is theoretically impossible.
Despite this limitation, we propose Conformal Prediction with Temporal
Dependence (CPTD), a procedure which is able to maintain strict cross-sectional
validity while improving longitudinal coverage. CPTD is post-hoc and
light-weight, and can easily be used in conjunction with any prediction model
as long as a calibration set is available. We focus on neural networks due to
their ability to model complicated data such as diagnosis codes for time-series
regression, and perform extensive experimental validation to verify the
efficacy of our approach. We find that CPTD outperforms baselines on a variety
of datasets by improving longitudinal coverage and often providing more
efficient (narrower) PIs.
Related papers
- TFT-multi: simultaneous forecasting of vital sign trajectories in the ICU [0.0]
Trajectory forecasting in healthcare data has been an important area of research in precision care and clinical integration for computational methods.
We extend the framework temporal fusion transformer (TFT), a multi-horizon time series prediction tool, and propose TFT-multi.
We apply TFT-multi to forecast 5 vital signs recorded in the intensive care unit: blood pressure, pulse, SpO2, temperature and respiratory rate.
arXiv Detail & Related papers (2024-09-23T22:36:37Z) - Anytime-Valid Inference for Double/Debiased Machine Learning of Causal Parameters [27.333679232669823]
Double (debiased) machine learning (DML) has seen widespread use in recent years for learning causal/structural parameters.
The classic double-debiased framework is only validally for a predetermined sample size.
This can be of particular concern in large scale experimental studies with huge financial costs or human lives at stake.
arXiv Detail & Related papers (2024-08-18T21:19:56Z) - Early Prediction of Causes (not Effects) in Healthcare by Long-Term Clinical Time Series Forecasting [11.96384267146423]
We propose to directly predict the causes via time series forecasting (TSF) of clinical variables.
Because model training does not rely on a particular label anymore, the forecasted data can be used to predict any consensus-based label.
arXiv Detail & Related papers (2024-08-07T14:52:06Z) - Mitigating LLM Hallucinations via Conformal Abstention [70.83870602967625]
We develop a principled procedure for determining when a large language model should abstain from responding in a general domain.
We leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate)
Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets.
arXiv Detail & Related papers (2024-04-04T11:32:03Z) - Conformal Predictions for Longitudinal Data [0.0]
We introduce Longitudinal Predictive Conformal Inference (L PCI), a distribution-free conformal prediction algorithm for longitudinal data.
Our experiments demonstrate that LPCI outperforms existing benchmarks in terms of longitudinal coverage rates.
The robust performance of LPCI in generating reliable prediction intervals for longitudinal data underscores its potential for broad applications.
arXiv Detail & Related papers (2023-10-04T14:51:07Z) - Conformal Prediction with Temporal Quantile Adjustments [40.282423098764404]
We develop a method to construct efficient and valid prediction intervals (PIs) for regression on cross-sectional time series data.
We validate TQA's performance through extensive experimentation.
arXiv Detail & Related papers (2022-05-20T03:31:03Z) - TACTiS: Transformer-Attentional Copulas for Time Series [76.71406465526454]
estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance.
We propose a versatile method that estimates joint distributions using an attention-based decoder.
We show that our model produces state-of-the-art predictions on several real-world datasets.
arXiv Detail & Related papers (2022-02-07T21:37:29Z) - Interval-censored Hawkes processes [82.87738318505582]
We propose a model to estimate the parameters of a Hawkes process in interval-censored settings.
We show how a non-homogeneous approximation to the Hawkes admits a tractable likelihood in the interval-censored setting.
arXiv Detail & Related papers (2021-04-16T07:29:04Z) - DeepRite: Deep Recurrent Inverse TreatmEnt Weighting for Adjusting
Time-varying Confounding in Modern Longitudinal Observational Data [68.29870617697532]
We propose Deep Recurrent Inverse TreatmEnt weighting (DeepRite) for time-varying confounding in longitudinal data.
DeepRite is shown to recover the ground truth from synthetic data, and estimate unbiased treatment effects from real data.
arXiv Detail & Related papers (2020-10-28T15:05:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.