Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources

• URL: http://arxiv.org/abs/2411.07414v1
• Date: Mon, 11 Nov 2024 22:36:50 GMT
• Title: Comparing Targeting Strategies for Maximizing Social Welfare with Limited Resources
• Authors: Vibhhu Sharma, Bryan Wilder,
• Abstract summary: We use data from 5 real-world RCTs in a variety of domains to empirically assess such choices. We find that risk-based targeting is almost always inferior to targeting based on even biased estimates of treatment effects.
• Score: 20.99198458867724
• License:
• Abstract: Machine learning is increasingly used to select which individuals receive limited-resource interventions in domains such as human services, education, development, and more. However, it is often not apparent what the right quantity is for models to predict. In particular, policymakers rarely have access to data from a randomized controlled trial (RCT) that would enable accurate estimates of treatment effects -- which individuals would benefit more from the intervention. Observational data is more likely to be available, creating a substantial risk of bias in treatment effect estimates. Practitioners instead commonly use a technique termed "risk-based targeting" where the model is just used to predict each individual's status quo outcome (an easier, non-causal task). Those with higher predicted risk are offered treatment. There is currently almost no empirical evidence to inform which choices lead to the most effect machine learning-informed targeting strategies in social domains. In this work, we use data from 5 real-world RCTs in a variety of domains to empirically assess such choices. We find that risk-based targeting is almost always inferior to targeting based on even biased estimates of treatment effects. Moreover, these results hold even when the policymaker has strong normative preferences for assisting higher-risk individuals. Our results imply that, despite the widespread use of risk prediction models in applied settings, practitioners may be better off incorporating even weak evidence about heterogeneous causal effects to inform targeting.

Related papers

• A Game-Theoretic Approach to Privacy-Utility Tradeoff in Sharing Genomic Summary Statistics [24.330984323956173]
  We propose a game-theoretic framework for optimal privacy-utility tradeoffs in the sharing of genomic summary statistics. Our experiments demonstrate that the proposed framework yields both stronger attacks and stronger defense strategies than the state of the art.
  arXiv  Detail & Related papers  (2024-06-03T22:09:47Z)
• Reduced-Rank Multi-objective Policy Learning and Optimization [57.978477569678844]
  In practice, causal researchers do not have a single outcome in mind a priori. In government-assisted social benefit programs, policymakers collect many outcomes to understand the multidimensional nature of poverty. We present a data-driven dimensionality-reduction methodology for multiple outcomes in the context of optimal policy learning.
  arXiv  Detail & Related papers  (2024-04-29T08:16:30Z)
• Defining Expertise: Applications to Treatment Effect Estimation [58.7977683502207]
  We argue that expertise - particularly the type of expertise the decision-makers of a domain are likely to have - can be informative in designing and selecting methods for treatment effect estimation. We define two types of expertise, predictive and prognostic, and demonstrate empirically that: (i) the prominent type of expertise in a domain significantly influences the performance of different methods in treatment effect estimation, and (ii) it is possible to predict the type of expertise present in a dataset.
  arXiv  Detail & Related papers  (2024-03-01T17:30:49Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• A New Approach for Interpretability and Reliability in Clinical Risk Prediction: Acute Coronary Syndrome Scenario [0.33927193323747895]
  We intend to create a new risk assessment methodology that combines the best characteristics of both risk score and machine learning models. The proposed approach achieved testing results identical to the standard LR, but offers superior interpretability and personalization. The reliability estimation of individual predictions presented a great correlation with the misclassifications rate.
  arXiv  Detail & Related papers  (2021-10-15T19:33:46Z)
• Impact of Interventional Policies Including Vaccine on Covid-19 Propagation and Socio-Economic Factors [0.7874708385247353]
  This study aims to provide a predictive analytics framework to model, predict and simulate COVID-19 propagation and socio-economic impact. We have leveraged a recently launched open-source COVID-19 big data platform and used published research to find potentially relevant variables. An advanced machine learning pipeline has been developed armed with a self-evolving model, deployed on a modern machine learning architecture.
  arXiv  Detail & Related papers  (2021-01-11T15:08:07Z)
• UNITE: Uncertainty-based Health Risk Prediction Leveraging Multi-sourced Data [81.00385374948125]
  We present UNcertaInTy-based hEalth risk prediction (UNITE) model. UNITE provides accurate disease risk prediction and uncertainty estimation leveraging multi-sourced health data. We evaluate UNITE on real-world disease risk prediction tasks: nonalcoholic fatty liver disease (NASH) and Alzheimer's disease (AD) UNITE achieves up to 0.841 in F1 score for AD detection, up to 0.609 in PR-AUC for NASH detection, and outperforms various state-of-the-art baselines by up to $19%$ over the best baseline.
  arXiv  Detail & Related papers  (2020-10-22T02:28:11Z)
• Epidemic mitigation by statistical inference from contact tracing data [61.04165571425021]
  We develop Bayesian inference methods to estimate the risk that an individual is infected. We propose to use probabilistic risk estimation in order to optimize testing and quarantining strategies for the control of an epidemic. Our approaches translate into fully distributed algorithms that only require communication between individuals who have recently been in contact.
  arXiv  Detail & Related papers  (2020-09-20T12:24:45Z)
• Enabling Counterfactual Survival Analysis with Balanced Representations [64.17342727357618]
  Survival data are frequently encountered across diverse medical applications, i.e., drug development, risk profiling, and clinical trials. We propose a theoretically grounded unified framework for counterfactual inference applicable to survival outcomes.
  arXiv  Detail & Related papers  (2020-06-14T01:15:00Z)
• Data-driven Simulation and Optimization for Covid-19 Exit Strategies [16.31545249131776]
  The rapid spread of the Coronavirus SARS-2 is a major challenge that led almost all governments worldwide to take drastic measures to respond to the tragedy. We have built a pandemic simulation and forecasting toolkit that combines a deep learning estimation of the epidemiological parameters of the disease.
  arXiv  Detail & Related papers  (2020-06-12T11:18:25Z)
• Regret Minimization for Causal Inference on Large Treatment Space [21.957539112375496]
  We propose a network architecture and a regularizer that extracts a debiased representation from biased observational data. Our proposed loss minimizes a classification error of whether or not the action is relatively good for the individual target.
  arXiv  Detail & Related papers  (2020-06-10T02:19:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.