Strategic Conformal Prediction

• URL: http://arxiv.org/abs/2411.01596v1
• Date: Sun, 03 Nov 2024 15:06:05 GMT
• Title: Strategic Conformal Prediction
• Authors: Daniel Csillag, Claudio José Struchiner, Guilherme Tegoni Goedert,
• Abstract summary: When a machine learning model is deployed, its predictions can alter its environment, as better informed agents strategize to suit their own interests. We propose a new framework, Strategic Conformal Prediction, which is capable of robust uncertainty quantification in such a setting.
• Score: 0.66567375919026
• License:
• Abstract: When a machine learning model is deployed, its predictions can alter its environment, as better informed agents strategize to suit their own interests. With such alterations in mind, existing approaches to uncertainty quantification break. In this work we propose a new framework, Strategic Conformal Prediction, which is capable of robust uncertainty quantification in such a setting. Strategic Conformal Prediction is backed by a series of theoretical guarantees spanning marginal coverage, training-conditional coverage, tightness and robustness to misspecification that hold in a distribution-free manner. Experimental analysis further validates our method, showing its remarkable effectiveness in face of arbitrary strategic alterations, whereas other methods break.

Related papers

• Uncertainty Quantification and Causal Considerations for Off-Policy Decision Making [4.514386953429771]
  Off-policy evaluation (OPE) seeks to assess the performance of a new policy using data collected under a different policy. Existing OPE methodologies suffer from several limitations arising from statistical uncertainty as well as causal considerations. We introduce the Marginal Ratio (MR) estimator, a novel OPE method that reduces variance by focusing on the marginal distribution of outcomes. Next, we propose Conformal Off-Policy Prediction (COPP), a principled approach for uncertainty quantification in OPE. Finally, we address causal unidentifiability in off-policy decision-making by developing novel bounds for sequential decision settings
  arXiv  Detail & Related papers  (2025-02-09T20:05:19Z)
• Calibrated Probabilistic Forecasts for Arbitrary Sequences [58.54729945445505]
  Real-world data streams can change unpredictably due to distribution shifts, feedback loops and adversarial actors. We present a forecasting framework ensuring valid uncertainty estimates regardless of how data evolves.
  arXiv  Detail & Related papers  (2024-09-27T21:46:42Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Ensembling Portfolio Strategies for Long-Term Investments: A Distribution-Free Preference Framework for Decision-Making and Algorithms [0.0]
  This paper investigates the problem of ensembling multiple strategies for sequential portfolios to outperform individual strategies in terms of long-term wealth. We introduce a novel framework for decision-making in combining strategies, irrespective of market conditions. We show results in favor of the proposed strategies, albeit with small tradeoffs in their Sharpe ratios.
  arXiv  Detail & Related papers  (2024-06-05T23:08:57Z)
• Conformal Prediction for Federated Uncertainty Quantification Under Label Shift [57.54977668978613]
  Federated Learning (FL) is a machine learning framework where many clients collaboratively train models. We develop a new conformal prediction method based on quantile regression and take into account privacy constraints.
  arXiv  Detail & Related papers  (2023-06-08T11:54:58Z)
• Conformal Off-Policy Prediction in Contextual Bandits [54.67508891852636]
  Conformal off-policy prediction can output reliable predictive intervals for the outcome under a new target policy. We provide theoretical finite-sample guarantees without making any additional assumptions beyond the standard contextual bandit setup.
  arXiv  Detail & Related papers  (2022-06-09T10:39:33Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Reliable Off-policy Evaluation for Reinforcement Learning [53.486680020852724]
  In a sequential decision-making problem, off-policy evaluation estimates the expected cumulative reward of a target policy. We propose a novel framework that provides robust and optimistic cumulative reward estimates using one or multiple logged data.
  arXiv  Detail & Related papers  (2020-11-08T23:16:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.