Conformal Predictive Systems Under Covariate Shift

• URL: http://arxiv.org/abs/2404.15018v2
• Date: Mon, 16 Sep 2024 10:32:28 GMT
• Title: Conformal Predictive Systems Under Covariate Shift
• Authors: Jef Jonkers, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke,
• Abstract summary: Conformal Predictive Systems (CPS) offer a versatile framework for constructing predictive distributions. We propose weighted CPS, akin to weighted Conformal Prediction (WCP) We present theoretical underpinnings and conjectures regarding the validity and efficacy of WCPS.
• Score: 2.9310590399782788
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conformal Predictive Systems (CPS) offer a versatile framework for constructing predictive distributions, allowing for calibrated inference and informative decision-making. However, their applicability has been limited to scenarios adhering to the Independent and Identically Distributed (IID) model assumption. This paper extends CPS to accommodate scenarios characterized by covariate shifts. We therefore propose Weighted CPS (WCPS), akin to Weighted Conformal Prediction (WCP), leveraging likelihood ratios between training and testing covariate distributions. This extension enables the construction of nonparametric predictive distributions capable of handling covariate shifts. We present theoretical underpinnings and conjectures regarding the validity and efficacy of WCPS and demonstrate its utility through empirical evaluations on both synthetic and real-world datasets. Our simulation experiments indicate that WCPS are probabilistically calibrated under covariate shift.

Related papers

• Model-free Methods for Event History Analysis and Efficient Adjustment (PhD Thesis) [55.2480439325792]
  This thesis is a series of independent contributions to statistics unified by a model-free perspective. The first chapter elaborates on how a model-free perspective can be used to formulate flexible methods that leverage prediction techniques from machine learning. The second chapter studies the concept of local independence, which describes whether the evolution of one process is directly influenced by another.
  arXiv  Detail & Related papers  (2025-02-11T19:24:09Z)
• Generalization and Informativeness of Weighted Conformal Risk Control Under Covariate Shift [40.43703709267958]
  Weighted conformal risk control (W-CRC) uses data collected during the training phase to convert point predictions into prediction sets with valid risk guarantees at test time. While W-CRC provides statistical reliability, its efficiency -- measured by the size of the prediction sets -- can only be assessed at test time.
  arXiv  Detail & Related papers  (2025-01-20T11:26:36Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Informativeness of Weighted Conformal Prediction [3.1717575440579617]
  We propose two methods to enhance the informativeness of weighted conformal prediction. We establish theoretical guarantees for our proposed methods and demonstrate their efficacy through simulations.
  arXiv  Detail & Related papers  (2024-05-10T13:55:08Z)
• COLEP: Certifiably Robust Learning-Reasoning Conformal Prediction via Probabilistic Circuits [21.140271657387903]
  Conformal prediction has shown spurring performance in constructing statistically rigorous prediction sets for arbitrary black-box machine learning models. We propose a certifiably robust learning-reasoning conformal prediction framework (COLEP) via probabilistic circuits. We show that COLEP achieves 12% up to improvement in certified coverage on GTSRB, 9% on CIFAR-10, and 14% on AwA2.
  arXiv  Detail & Related papers  (2024-03-17T21:23:45Z)
• Federated Conformal Predictors for Distributed Uncertainty Quantification [83.50609351513886]
  Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning. In this paper, we extend conformal prediction to the federated learning setting. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction framework.
  arXiv  Detail & Related papers  (2023-05-27T19:57:27Z)
• Learning Counterfactually Invariant Predictors [11.682403472580162]
  We propose a model-agnostic framework, called Counterfactually Invariant Prediction (CIP) Our experimental results demonstrate the effectiveness of CIP in enforcing counterfactual invariance across various simulated and real-world datasets.
  arXiv  Detail & Related papers  (2022-07-20T09:23:35Z)
• Probabilistic Conformal Prediction Using Conditional Random Samples [73.26753677005331]
  PCP is a predictive inference algorithm that estimates a target variable by a discontinuous predictive set. It is efficient and compatible with either explicit or implicit conditional generative models.
  arXiv  Detail & Related papers  (2022-06-14T03:58:03Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)
• Repulsive Mixture Models of Exponential Family PCA for Clustering [127.90219303669006]
  The mixture extension of exponential family principal component analysis ( EPCA) was designed to encode much more structural information about data distribution than the traditional EPCA. The traditional mixture of local EPCAs has the problem of model redundancy, i.e., overlaps among mixing components, which may cause ambiguity for data clustering. In this paper, a repulsiveness-encouraging prior is introduced among mixing components and a diversified EPCA mixture (DEPCAM) model is developed in the Bayesian framework.
  arXiv  Detail & Related papers  (2020-04-07T04:07:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.