Enhancing reliability in prediction intervals using point forecasters: Heteroscedastic Quantile Regression and Width-Adaptive Conformal Inference

• URL: http://arxiv.org/abs/2406.14904v1
• Date: Fri, 21 Jun 2024 06:51:13 GMT
• Title: Enhancing reliability in prediction intervals using point forecasters: Heteroscedastic Quantile Regression and Width-Adaptive Conformal Inference
• Authors: Carlos Sebastián, Carlos E. González-Guillén, Jesús Juan,
• Abstract summary: We argue that, when evaluating a set of intervals, traditional measures alone are insufficient. The intervals must vary in length, with this variation directly linked to the difficulty of the prediction. We propose the Heteroscedastic Quantile Regression (HQR) model and the Width-Adaptive Conformal Inference ( WACI) method.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Building prediction intervals for time series forecasting problems presents a complex challenge, particularly when relying solely on point predictors, a common scenario for practitioners in the industry. While research has primarily focused on achieving increasingly efficient valid intervals, we argue that, when evaluating a set of intervals, traditional measures alone are insufficient. There are additional crucial characteristics: the intervals must vary in length, with this variation directly linked to the difficulty of the prediction, and the coverage of the interval must remain independent of the difficulty of the prediction for practical utility. We propose the Heteroscedastic Quantile Regression (HQR) model and the Width-Adaptive Conformal Inference (WACI) method, providing theoretical coverage guarantees, to overcome those issues, respectively. The methodologies are evaluated in the context of Electricity Price Forecasting and Wind Power Forecasting, representing complex scenarios in time series forecasting. The results demonstrate that HQR and WACI not only improve or achieve typical measures of validity and efficiency but also successfully fulfil the commonly ignored mentioned characteristics.

Related papers

• Conformal Prediction Algorithms for Time Series Forecasting: Methods and Benchmarking [0.0]
  Time series temporal dependencies violate the core assumption of data exchangeability.<n>This paper critically examines the main categories of algorithmic solutions designed to address this conflict.<n>We use AutoARIMA as the base forecaster on a large-scale monthly sales dataset.
  arXiv  Detail & Related papers  (2026-01-26T14:15:08Z)
• Fast Conformal Prediction using Conditional Interquantile Intervals [9.881784717196675]
  We introduce Conformal Interquantile Regression (CIR), a conformal regression method that constructs near-minimal prediction intervals with guaranteed coverage.<n>We also propose CIR+, which enhances CIR by incorporating a width-based selection rule for interquantile intervals.
  arXiv  Detail & Related papers  (2026-01-06T07:01:08Z)
• A Gentle Introduction to Conformal Time Series Forecasting [0.0]
  This review unifies recent advances in conformal forecasting methods specifically designed to address nonexchangeable data.<n>We first present a theoretical foundation, deriving finite-sample guarantees for split-conformal prediction under mild weak-dependence conditions.<n>We then survey and classify state-of-the-art approaches that mitigate serial dependence by reweighting calibration data, dynamically updating residual distributions, or adaptively tuning target coverage levels in real time.
  arXiv  Detail & Related papers  (2025-11-17T17:12:51Z)
• Predictive inference for time series: why is split conformal effective despite temporal dependence? [8.032656343027146]
  Conformal prediction methods provide distribution-free coverage for any iid or exchangeable data distribution.<n>Using predictors with "memory" -- i.e., predictors that utilize past observations, such as autoregressive models -- further exacerbates this problem.<n>Our results bound the loss of coverage of these methods in terms of a new "switch coefficient", measuring the extent to which temporal dependence within the time series creates violations of exchangeability.
  arXiv  Detail & Related papers  (2025-10-02T18:24:04Z)
• Adaptive Conformal Prediction Intervals Over Trajectory Ensembles [50.31074512684758]
  Future trajectories play an important role across domains such as autonomous driving, hurricane forecasting, and epidemic modeling.<n>We propose a unified framework based on conformal prediction that transforms sampled trajectories into calibrated prediction intervals with theoretical coverage guarantees.
  arXiv  Detail & Related papers  (2025-08-18T21:14:07Z)
• Topology-Aware Conformal Prediction for Stream Networks [54.505880918607296]
  We propose Spatio-Temporal Adaptive Conformal Inference (textttCISTA), a novel framework that integrates network topology and temporal dynamics into the conformal prediction framework. Our results show that textttCISTA effectively balances prediction efficiency and coverage, outperforming existing conformal prediction methods for stream networks.
  arXiv  Detail & Related papers  (2025-03-06T21:21:15Z)
• Relational Conformal Prediction for Correlated Time Series [56.59852921638328]
  We propose a novel distribution-free approach based on conformal prediction framework and quantile regression. We fill this void by introducing a novel conformal prediction method based on graph deep learning operators. Our approach provides accurate coverage and archives state-of-the-art uncertainty quantification in relevant benchmarks.
  arXiv  Detail & Related papers  (2025-02-13T16:12:17Z)
• Conformal Inference of Individual Treatment Effects Using Conditional Density Estimates [3.7307776333361122]
  Current state-of-the-art approaches, while providing valid prediction intervals, often yield overly conservative prediction intervals. In this work, we introduce a conformal inference approach to ITE using the conditional density of the outcome. We show that our prediction intervals are not only marginally valid but are narrower than existing methods.
  arXiv  Detail & Related papers  (2025-01-24T21:46:37Z)
• Deconfounding Time Series Forecasting [1.5967186772129907]
  Time series forecasting is a critical task in various domains, where accurate predictions can drive informed decision-making. Traditional forecasting methods often rely on current observations of variables to predict future outcomes. We propose an enhanced forecasting approach that incorporates representations of latent confounders derived from historical data.
  arXiv  Detail & Related papers  (2024-10-27T12:45:42Z)
• JANET: Joint Adaptive predictioN-region Estimation for Time-series [28.19630729432862]
  We propose JANET (Joint Adaptive predictioN-region Estimation for Time-series), a novel framework for constructing conformal prediction regions. JANET generalises the inductive conformal framework and efficiently produces joint prediction regions with controlled K-familywise error rates. Our empirical evaluation demonstrates JANET's superior performance in multi-step prediction tasks across diverse time series datasets.
  arXiv  Detail & Related papers  (2024-07-08T21:03:15Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
  Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs. We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint. We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
  arXiv  Detail & Related papers  (2024-06-05T13:36:38Z)
• Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification [59.81904428056924]
  We introduce SMASH: a Score MAtching estimator for learning markedPs with uncertainty quantification. Specifically, our framework adopts a normalization-free objective by estimating the pseudolikelihood of markedPs through score-matching. The superior performance of our proposed framework is demonstrated through extensive experiments in both event prediction and uncertainty quantification.
  arXiv  Detail & Related papers  (2023-10-25T02:37:51Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• On the Expected Size of Conformal Prediction Sets [24.161372736642157]
  We theoretically quantify the expected size of the prediction sets under the split conformal prediction framework. As this precise formulation cannot usually be calculated directly, we derive point estimates and high-probability bounds interval. We corroborate the efficacy of our results with experiments on real-world datasets for both regression and classification problems.
  arXiv  Detail & Related papers  (2023-06-12T17:22:57Z)
• Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts [73.33395097728128]
  We provide the first systematic finite-sample study of proper scoring rules for time-series forecasting evaluation. We carry out our analysis on a comprehensive synthetic benchmark, specifically designed to test several key discrepancies between ground-truth and forecast distributions.
  arXiv  Detail & Related papers  (2023-04-19T17:38:42Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Distribution-Free Finite-Sample Guarantees and Split Conformal Prediction [0.0]
  split conformal prediction represents a promising avenue to obtain finite-sample guarantees under minimal distribution-free assumptions. We highlight the connection between split conformal prediction and classical tolerance predictors developed in the 1940s.
  arXiv  Detail & Related papers  (2022-10-26T14:12:24Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Conformal prediction set for time-series [16.38369532102931]
  Uncertainty quantification is essential to studying complex machine learning methods. We develop Ensemble Regularized Adaptive Prediction Set (ERAPS) to construct prediction sets for time-series. We show valid marginal and conditional coverage by ERAPS, which also tends to yield smaller prediction sets than competing methods.
  arXiv  Detail & Related papers  (2022-06-15T23:48:53Z)
• Quantifying Uncertainty in Deep Spatiotemporal Forecasting [67.77102283276409]
  We describe two types of forecasting problems: regular grid-based and graph-based. We analyze UQ methods from both the Bayesian and the frequentist point view, casting in a unified framework via statistical decision theory. Through extensive experiments on real-world road network traffic, epidemics, and air quality forecasting tasks, we reveal the statistical computational trade-offs for different UQ methods.
  arXiv  Detail & Related papers  (2021-05-25T14:35:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.