Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction

• URL: http://arxiv.org/abs/2607.02206v1
• Date: Thu, 02 Jul 2026 14:13:08 GMT
• Title: Prediction Sets for Counterfactual Decisions: Coverage, Optimality, and Conformal Prediction
• Abstract summary: We develop a decision-theoretic framework for uncertainty-informed counterfactual decisions.<n>We identify a novel notion of emphpolicy-coupled coverage -- namely, coverage of the realized outcome under the action induced by the prediction sets themselves.
• Score: 7.899668963928307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predictions are increasingly used to guide high-stakes decisions, from treatment selection to policy making. To ensure reliability with imperfect predictions, uncertainty quantification methods such as conformal prediction build prediction sets with coverage guarantees. However, statistical validity alone does not immediately determine the decisions to take, nor the optimality thereof. This gap is especially delicate in counterfactual settings where the outcome that materializes depends on the action taken, so uncertainty cannot be specified independently of the decision rule. We develop a decision-theoretic framework for uncertainty-informed counterfactual decisions. We identify a novel notion of \emph{policy-coupled coverage} -- namely, coverage of the realized outcome under the action induced by the prediction sets themselves -- as the optimal and lossless interface between uncertainty and action. It plays three roles. First, it justifies acting via a natural max-min rule as minimax-optimal under distributional ambiguity. Second, optimizing prediction sets under policy-coupled coverage is equivalent both to a stronger universal-coverage formulation and to the direct risk-averse optimization over policies and utility certificates; this equivalence yields the explicit form of the population-optimal prediction sets. Third, it admits a two-stage procedure, Policy-Coupled Risk-Averse Conformal Prediction (PC-RACP), that approximates these optimal sets with rigorous finite-sample coverage. Simulations and a real email-marketing experiment confirm that PC-RACP delivers higher utility than existing approaches while maintaining valid coverage, and that ignoring the counterfactual structure of the decision problem is suboptimal for both validity and utility.

Related papers

• Optimal Decision-Making Based on Prediction Sets [18.860889057545467]
  Prediction sets can wrap around any ML model to cover unknown test outcomes with a guaranteed probability.<n>We propose a decision-theoretic framework that seeks to minimize the expected loss (risk) against a worst-case distribution consistent with the prediction set's coverage guarantee.
  arXiv  Detail & Related papers  (2026-02-01T03:02:44Z)
• Distribution-informed Online Conformal Prediction [53.674678995825666]
  We propose Conformal Optimistic Prediction (COP), an online conformal prediction algorithm incorporating underlying data pattern into the update rule.<n>COP produces tighter prediction sets when predictable pattern exists, while retaining valid coverage guarantees even when estimates are inaccurate.<n>We prove that COP can achieve valid coverage and construct shorter prediction intervals than other baselines.
  arXiv  Detail & Related papers  (2025-12-08T17:51:49Z)
• When Robustness Meets Conservativeness: Conformalized Uncertainty Calibration for Balanced Decision Making [8.234618636958462]
  We propose a new framework that provides distribution-free, finite-sample guarantees on miscoverage and regret.<n>Our method constructs valid estimators that trace out the mis-regret frontier.<n>These results offer the first principled data-driven methodology for guiding robustness selection.
  arXiv  Detail & Related papers  (2025-10-09T03:38:17Z)
• A Principled Approach to Randomized Selection under Uncertainty: Applications to Peer Review and Grant Funding [61.86327960322782]
  We propose a principled framework for randomized decision-making based on interval estimates of the quality of each item.<n>We introduce MERIT, an optimization-based method that maximizes the worst-case expected number of top candidates selected.<n>We prove that MERIT satisfies desirable axiomatic properties not guaranteed by existing approaches.
  arXiv  Detail & Related papers  (2025-06-23T19:59:30Z)
• Optimal Conformal Prediction under Epistemic Uncertainty [61.46247583794497]
  Conformal prediction (CP) is a popular framework for representing uncertainty.<n>We introduce Bernoulli prediction sets (BPS) which produce the smallest prediction sets that ensure conditional coverage.<n>When given first-order predictions, BPS reduces to the well-known adaptive prediction sets (APS)
  arXiv  Detail & Related papers  (2025-05-25T08:32:44Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agents [24.938391962245877]
  We develop decision-theoretic foundations that connect uncertainty using prediction sets with risk-averse decision-making.<n>We experimentally demonstrate the significant advantages of Risk-Averse (RAC) in applications such as medical diagnosis and recommendation systems.
  arXiv  Detail & Related papers  (2025-02-04T18:37:10Z)
• Conformal Contextual Robust Optimization [21.2737854880866]
  Data-driven approaches to predict probabilistic decision-making problems seek to mitigate the risk of uncertainty region mis robustness in safety-critical settings. We propose a Conformal-Then-Predict (CPO) framework for. probability-then-optimize decision-making problems.
  arXiv  Detail & Related papers  (2023-10-16T01:58:27Z)
• Conformal Off-Policy Prediction in Contextual Bandits [54.67508891852636]
  Conformal off-policy prediction can output reliable predictive intervals for the outcome under a new target policy. We provide theoretical finite-sample guarantees without making any additional assumptions beyond the standard contextual bandit setup.
  arXiv  Detail & Related papers  (2022-06-09T10:39:33Z)
• Off-Policy Evaluation with Policy-Dependent Optimization Response [90.28758112893054]
  We develop a new framework for off-policy evaluation with a textitpolicy-dependent linear optimization response. We construct unbiased estimators for the policy-dependent estimand by a perturbation method. We provide a general algorithm for optimizing causal interventions.
  arXiv  Detail & Related papers  (2022-02-25T20:25:37Z)
• Conformal Uncertainty Sets for Robust Optimization [0.0]
  We use Mahalanobis distance as a novel function for multi-target regression and the construction of joint prediction regions. We also connect conformal prediction regions to robust optimization, providing finite sample valid and conservative uncertainty sets.
  arXiv  Detail & Related papers  (2021-05-31T13:42:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.