Conformal Prediction for Hierarchical Data

• URL: http://arxiv.org/abs/2411.13479v1
• Date: Wed, 20 Nov 2024 17:26:26 GMT
• Title: Conformal Prediction for Hierarchical Data
• Authors: Guillaume Principato, Yvenn Amara-Ouali, Yannig Goude, Bachir Hamrouche, Jean-Michel Poggi, Gilles Stoltz,
• Abstract summary: We propose a first step towards combining Conformal Prediction and Forecast Reconciliation. We show that the validity granted by SCP remains while improving the efficiency of the prediction sets.
• Score: 5.580128181112309
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconciliation has become an essential tool in multivariate point forecasting for hierarchical time series. However, there is still a lack of understanding of the theoretical properties of probabilistic Forecast Reconciliation techniques. Meanwhile, Conformal Prediction is a general framework with growing appeal that provides prediction sets with probabilistic guarantees in finite sample. In this paper, we propose a first step towards combining Conformal Prediction and Forecast Reconciliation by analyzing how including a reconciliation step in the Split Conformal Prediction (SCP) procedure enhances the resulting prediction sets. In particular, we show that the validity granted by SCP remains while improving the efficiency of the prediction sets. We also advocate a variation of the theoretical procedure for practical use. Finally, we illustrate these results with simulations.

Related papers

• Multivariate Conformal Prediction using Optimal Transport [19.644272536912286]
  Conformal prediction (CP) quantifies the uncertainty of machine learning models. We propose a principled framework for constructing conformal prediction sets in multidimensional settings.
  arXiv  Detail & Related papers  (2025-02-05T20:56:41Z)
• Calibrated Probabilistic Forecasts for Arbitrary Sequences [58.54729945445505]
  Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors. We present a forecasting framework ensuring valid uncertainty estimates regardless of how data evolves.
  arXiv  Detail & Related papers  (2024-09-27T21:46:42Z)
• Ranking and Combining Latent Structured Predictive Scores without Labeled Data [2.5064967708371553]
  This paper introduces a novel structured unsupervised ensemble learning model (SUEL) It exploits the dependency between a set of predictors with continuous predictive scores, rank the predictors without labeled data and combine them to an ensembled score with weights. The efficacy of the proposed methods is rigorously assessed through both simulation studies and real-world application of risk genes discovery.
  arXiv  Detail & Related papers  (2024-08-14T20:14:42Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Structured Prediction in Online Learning [66.36004256710824]
  We study a theoretical and algorithmic framework for structured prediction in the online learning setting. We show that our algorithm is a generalisation of optimal algorithms from the supervised learning setting. We consider a second algorithm designed especially for non-stationary data distributions, including adversarial data.
  arXiv  Detail & Related papers  (2024-06-18T07:45:02Z)
• Conformalized Late Fusion Multi-View Learning [18.928543069018865]
  Uncertainty quantification for multi-view learning is motivated by the increasing use of multi-view data in scientific problems. A common variant of multi-view learning is late fusion: train separate predictors on individual views and combine them after single-view predictions are available. We propose a novel methodology, Multi-View Conformal Prediction (MVCP), where conformal prediction is instead performed separately on the single-view predictors and only fused subsequently.
  arXiv  Detail & Related papers  (2024-05-25T14:11:01Z)
• Conformal Prediction with Learned Features [22.733758606168873]
  We propose Partition Learning Conformal Prediction (PLCP) to improve conditional validity of prediction sets. We implement PLCP efficiently with gradient alternating descent, utilizing off-the-shelf machine learning models. Our experimental results over four real-world and synthetic datasets show the superior performance of PLCP.
  arXiv  Detail & Related papers  (2024-04-26T15:43:06Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• On the Expected Size of Conformal Prediction Sets [24.161372736642157]
  We theoretically quantify the expected size of the prediction sets under the split conformal prediction framework. As this precise formulation cannot usually be calculated directly, we derive point estimates and high-probability bounds interval. We corroborate the efficacy of our results with experiments on real-world datasets for both regression and classification problems.
  arXiv  Detail & Related papers  (2023-06-12T17:22:57Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Predictive Inference with Feature Conformal Prediction [80.77443423828315]
  We propose feature conformal prediction, which extends the scope of conformal prediction to semantic feature spaces. From a theoretical perspective, we demonstrate that feature conformal prediction provably outperforms regular conformal prediction under mild assumptions. Our approach could be combined with not only vanilla conformal prediction, but also other adaptive conformal prediction methods.
  arXiv  Detail & Related papers  (2022-10-01T02:57:37Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Conformal prediction set for time-series [16.38369532102931]
  Uncertainty quantification is essential to studying complex machine learning methods. We develop Ensemble Regularized Adaptive Prediction Set (ERAPS) to construct prediction sets for time-series. We show valid marginal and conditional coverage by ERAPS, which also tends to yield smaller prediction sets than competing methods.
  arXiv  Detail & Related papers  (2022-06-15T23:48:53Z)
• Probabilistic Conformal Prediction Using Conditional Random Samples [73.26753677005331]
  PCP is a predictive inference algorithm that estimates a target variable by a discontinuous predictive set. It is efficient and compatible with either explicit or implicit conditional generative models.
  arXiv  Detail & Related papers  (2022-06-14T03:58:03Z)
• Aggregating distribution forecasts from deep ensembles [0.0]
  We propose a general quantile aggregation framework for deep ensembles. We show that combining forecast distributions from deep ensembles can substantially improve the predictive performance.
  arXiv  Detail & Related papers  (2022-04-05T15:42:51Z)
• Conformal prediction for time series [16.38369532102931]
  textttEnbPI wraps around ensemble predictors, which is closely related to conformal prediction (CP) but does not require data exchangeability. We perform extensive simulation and real-data analyses to demonstrate its effectiveness compared with existing methods.
  arXiv  Detail & Related papers  (2020-10-18T21:05:32Z)
• Learning Output Embeddings in Structured Prediction [73.99064151691597]
  A powerful and flexible approach to structured prediction consists in embedding the structured objects to be predicted into a feature space of possibly infinite dimension. A prediction in the original space is computed by solving a pre-image problem. In this work, we propose to jointly learn a finite approximation of the output embedding and the regression function into the new feature space.
  arXiv  Detail & Related papers  (2020-07-29T09:32:53Z)
• Hierarchical forecast reconciliation with machine learning [0.34998703934432673]
  This paper proposes a novel hierarchical forecasting approach based on machine learning. It structurally combines the objectives of improved post-sample empirical forecasting accuracy and coherence. Our results suggest that the proposed method gives superior point forecasts than existing approaches.
  arXiv  Detail & Related papers  (2020-06-03T04:49:39Z)
• A General Framework for Consistent Structured Prediction with Implicit Loss Embeddings [113.15416137912399]
  We propose and analyze a novel theoretical and algorithmic framework for structured prediction. We study a large class of loss functions that implicitly defines a suitable geometry on the problem. When dealing with output spaces with infinite cardinality, a suitable implicit formulation of the estimator is shown to be crucial.
  arXiv  Detail & Related papers  (2020-02-13T10:30:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.