Statistical Decision Theory with Counterfactual Loss

• URL: http://arxiv.org/abs/2505.08908v1
• Date: Tue, 13 May 2025 19:00:07 GMT
• Title: Statistical Decision Theory with Counterfactual Loss
• Authors: Benedikt Koch, Kosuke Imai,
• Abstract summary: We extend standard decision theory to incorporate counterfactual losses--criteria that evaluate decisions using all potential outcomes.<n>We show that under the assumption of strong ignorability, a counterfactual risk is identifiable if and only if the counterfactual loss function is additive in the potential outcomes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Classical statistical decision theory evaluates treatment choices based solely on observed outcomes. However, by ignoring counterfactual outcomes, it cannot assess the quality of decisions relative to feasible alternatives. For example, the quality of a physician's decision may depend not only on patient survival, but also on whether a less invasive treatment could have produced a similar result. To address this limitation, we extend standard decision theory to incorporate counterfactual losses--criteria that evaluate decisions using all potential outcomes. The central challenge in this generalization is identification: because only one potential outcome is observed for each unit, the associated risk under a counterfactual loss is generally not identifiable. We show that under the assumption of strong ignorability, a counterfactual risk is identifiable if and only if the counterfactual loss function is additive in the potential outcomes. Moreover, we demonstrate that additive counterfactual losses can yield treatment recommendations that differ from those based on standard loss functions, provided that the decision problem involves more than two treatment options.

Related papers

• Uncertainty Quantification for Regression using Proper Scoring Rules [76.24649098854219]
  We introduce a unified UQ framework for regression based on proper scoring rules, such as CRPS, logarithmic, squared error, and quadratic scores.<n>We derive closed-form expressions for the uncertainty measures under practical parametric assumptions and show how to estimate them using ensembles of models.<n>Our broad evaluation on synthetic and real-world regression datasets provides guidance for selecting reliable UQ measures.
  arXiv  Detail & Related papers  (2025-09-30T17:52:12Z)
• Uncertainty Quantification with Proper Scoring Rules: Adjusting Measures to Prediction Tasks [19.221081896134567]
  We propose measures of uncertainty based on a known decomposition of (strictly) proper scoring rules, a specific type of loss function, into a divergence and an entropy component.<n>This leads to a flexible framework for uncertainty quantification that can be instantiated with different losses (scoring rules)<n>We show that this flexibility is indeed advantageous. In particular, we analyze the task of selective prediction and show that the scoring rule should ideally match the task loss.
  arXiv  Detail & Related papers  (2025-05-28T16:22:53Z)
• Marginal Fairness: Fair Decision-Making under Risk Measures [24.99817090886293]
  This paper introduces marginal fairness, a new individual fairness notion for equitable decision-making in the presence of protected attributes.<n>We model business decision-making in highly regulated industries (such as insurance and finance) as a two-step process.<n>A numerical study and an empirical implementation using an auto insurance dataset demonstrate how the framework can be applied in practice.
  arXiv  Detail & Related papers  (2025-05-24T22:44:35Z)
• Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learner [72.20769640318969]
  Estimating causal quantities from observational data is crucial for understanding the safety and effectiveness of medical treatments.<n>Medical practitioners require not only estimating averaged causal quantities, but also understanding the randomness of the treatment effect as a random variable.<n>This randomness is referred to as aleatoric uncertainty and is necessary for understanding the probability of benefit from treatment or quantiles of the treatment effect.
  arXiv  Detail & Related papers  (2024-11-05T18:14:49Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users.<n>We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions.<n>We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Fairness-Accuracy Trade-Offs: A Causal Perspective [58.06306331390586]
  We analyze the tension between fairness and accuracy from a causal lens for the first time.<n>We show that enforcing a causal constraint often reduces the disparity between demographic groups.<n>We introduce a new neural approach for causally-constrained fair learning.
  arXiv  Detail & Related papers  (2024-05-24T11:19:52Z)
• Auditing Fairness under Unobserved Confounding [56.61738581796362]
  We show that, surprisingly, one can still compute meaningful bounds on treatment rates for high-risk individuals.<n>We use the fact that in many real-world settings we have data from prior to any allocation to derive unbiased estimates of risk.
  arXiv  Detail & Related papers  (2024-03-18T21:09:06Z)
• Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding [2.8498944632323755]
  We propose a unified framework for the robust design and evaluation of predictive algorithms in selectively observed data. We impose general assumptions on how much the outcome may vary on average between unselected and selected units. We develop debiased machine learning estimators for the bounds on a large class of predictive performance estimands.
  arXiv  Detail & Related papers  (2022-12-19T20:41:44Z)
• Bounding Counterfactuals under Selection Bias [60.55840896782637]
  We propose a first algorithm to address both identifiable and unidentifiable queries. We prove that, in spite of the missingness induced by the selection bias, the likelihood of the available data is unimodal.
  arXiv  Detail & Related papers  (2022-07-26T10:33:10Z)
• Policy Learning with Asymmetric Counterfactual Utilities [0.6138671548064356]
  We consider optimal policy learning with asymmetric counterfactual utility functions. We derive minimax decision rules by minimizing the maximum expected utility loss. We show that one can learn minimax loss decision rules from observed data by solving intermediate classification problems.
  arXiv  Detail & Related papers  (2022-06-21T15:44:49Z)
• Robust and Agnostic Learning of Conditional Distributional Treatment Effects [62.44901952244514]
  The conditional average treatment effect (CATE) is the best point prediction of individual causal effects. In aggregate analyses, this is usually addressed by measuring distributional treatment effect (DTE) We provide a new robust and model-agnostic methodology for learning the conditional DTE (CDTE) for a wide class of problems.
  arXiv  Detail & Related papers  (2022-05-23T17:40:31Z)
• To Impute or not to Impute? -- Missing Data in Treatment Effect Estimation [84.76186111434818]
  We identify a new missingness mechanism, which we term mixed confounded missingness (MCM), where some missingness determines treatment selection and other missingness is determined by treatment selection. We show that naively imputing all data leads to poor performing treatment effects models, as the act of imputation effectively removes information necessary to provide unbiased estimates. Our solution is selective imputation, where we use insights from MCM to inform precisely which variables should be imputed and which should not.
  arXiv  Detail & Related papers  (2022-02-04T12:08:31Z)
• Treatment Effect Risk: Bounds and Inference [58.442274475425144]
  Since the average treatment effect measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10% of the population. In this paper we consider how to nonetheless assess this important risk measure, formalized as the conditional value at risk (CVaR) of the ITE distribution. Some bounds can also be interpreted as summarizing a complex CATE function into a single metric and are of interest independently of being a bound.
  arXiv  Detail & Related papers  (2022-01-15T17:21:26Z)
• Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving [1.6758573326215689]
  Reinforcement learning can be used to create a decision-making agent for autonomous driving. Previous approaches provide only black-box solutions, which do not offer information on how confident the agent is about its decisions. This paper introduces the Ensemble Quantile Networks (EQN) method, which combines distributional RL with an ensemble approach to obtain a complete uncertainty estimate.
  arXiv  Detail & Related papers  (2021-05-21T10:36:16Z)
• Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression [91.3373131262391]
  Uncertainty is the only certainty there is. Traditionally, the direct regression formulation is considered and the uncertainty is modeled by modifying the output space to a certain family of probabilistic distributions. How to model the uncertainty within the present-day technologies for regression remains an open issue.
  arXiv  Detail & Related papers  (2021-03-25T06:56:09Z)
• Estimating Average Treatment Effects via Orthogonal Regularization [18.586616164230566]
  Previous methods estimate outcomes based on unconfoundedness but neglect any constraints that unconfoundedness imposes on the outcomes. We propose a novel regularization framework for estimating average treatment effects that exploits unconfoundedness. We demonstrate that DONUT outperforms the state-of-the-art substantially.
  arXiv  Detail & Related papers  (2021-01-21T08:05:35Z)
• Enabling Counterfactual Survival Analysis with Balanced Representations [64.17342727357618]
  Survival data are frequently encountered across diverse medical applications, i.e., drug development, risk profiling, and clinical trials. We propose a theoretically grounded unified framework for counterfactual inference applicable to survival outcomes.
  arXiv  Detail & Related papers  (2020-06-14T01:15:00Z)
• Off-policy Policy Evaluation For Sequential Decisions Under Unobserved Confounding [33.58862183373374]
  We assess robustness of OPE methods under unobserved confounding. We show that even small amounts of per-decision confounding can heavily bias OPE methods. We propose an efficient loss-minimization-based procedure for computing worst-case bounds.
  arXiv  Detail & Related papers  (2020-03-12T05:20:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.