Relative Sparsity for Medical Decision Problems
- URL: http://arxiv.org/abs/2211.16566v3
- Date: Fri, 31 Mar 2023 13:31:33 GMT
- Title: Relative Sparsity for Medical Decision Problems
- Authors: Samuel J. Weisenthal, Sally W. Thurston, Ashkan Ertefaie
- Abstract summary: It is often important to explain to the healthcare provider, and to the patient, how a new policy differs from the current standard of care.
We propose a criterion for selecting $lambda$, perform simulations, and illustrate our method with a real, observational healthcare dataset.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Existing statistical methods can estimate a policy, or a mapping from
covariates to decisions, which can then instruct decision makers (e.g., whether
to administer hypotension treatment based on covariates blood pressure and
heart rate). There is great interest in using such data-driven policies in
healthcare. However, it is often important to explain to the healthcare
provider, and to the patient, how a new policy differs from the current
standard of care. This end is facilitated if one can pinpoint the aspects of
the policy (i.e., the parameters for blood pressure and heart rate) that change
when moving from the standard of care to the new, suggested policy. To this
end, we adapt ideas from Trust Region Policy Optimization (TRPO). In our work,
however, unlike in TRPO, the difference between the suggested policy and
standard of care is required to be sparse, aiding with interpretability. This
yields ``relative sparsity," where, as a function of a tuning parameter,
$\lambda$, we can approximately control the number of parameters in our
suggested policy that differ from their counterparts in the standard of care
(e.g., heart rate only). We propose a criterion for selecting $\lambda$,
perform simulations, and illustrate our method with a real, observational
healthcare dataset, deriving a policy that is easy to explain in the context of
the current standard of care. Our work promotes the adoption of data-driven
decision aids, which have great potential to improve health outcomes.
Related papers
- Policy Learning with Distributional Welfare [1.0742675209112622]
Most literature on treatment choice has considered utilitarian welfare based on the conditional average treatment effect (ATE)
This paper proposes an optimal policy that allocates the treatment based on the conditional quantile of individual treatment effects (QoTE)
arXiv Detail & Related papers (2023-11-27T14:51:30Z) - Inference for relative sparsity [0.0]
We develop inference for the relative sparsity objective function, because characterizing uncertainty is crucial to applications in medicine.
Inference is difficult, because the relative sparsity objective depends on the unpenalized value function, which is unstable and has infinite estimands in the binary action case.
To tackle these issues, we nest a weighted Trust Region Policy Optimization function within a relative sparsity objective, implement an adaptive relative sparsity penalty, and propose a sample-splitting framework for post-selection inference.
arXiv Detail & Related papers (2023-06-25T17:14:45Z) - Conformal Off-Policy Evaluation in Markov Decision Processes [53.786439742572995]
Reinforcement Learning aims at identifying and evaluating efficient control policies from data.
Most methods for this learning task, referred to as Off-Policy Evaluation (OPE), do not come with accuracy and certainty guarantees.
We present a novel OPE method based on Conformal Prediction that outputs an interval containing the true reward of the target policy with a prescribed level of certainty.
arXiv Detail & Related papers (2023-04-05T16:45:11Z) - Evaluating COVID-19 vaccine allocation policies using Bayesian $m$-top
exploration [53.122045119395594]
We present a novel technique for evaluating vaccine allocation strategies using a multi-armed bandit framework.
$m$-top exploration allows the algorithm to learn $m$ policies for which it expects the highest utility.
We consider the Belgian COVID-19 epidemic using the individual-based model STRIDE, where we learn a set of vaccination policies.
arXiv Detail & Related papers (2023-01-30T12:22:30Z) - Policy Learning with Asymmetric Counterfactual Utilities [0.6138671548064356]
We consider optimal policy learning with asymmetric counterfactual utility functions.
We derive minimax decision rules by minimizing the maximum expected utility loss.
We show that one can learn minimax loss decision rules from observed data by solving intermediate classification problems.
arXiv Detail & Related papers (2022-06-21T15:44:49Z) - Optimal discharge of patients from intensive care via a data-driven
policy learning framework [58.720142291102135]
It is important that the patient discharge task addresses the nuanced trade-off between decreasing a patient's length of stay and the risk of readmission or even death following the discharge decision.
This work introduces an end-to-end general framework for capturing this trade-off to recommend optimal discharge timing decisions.
A data-driven approach is used to derive a parsimonious, discrete state space representation that captures a patient's physiological condition.
arXiv Detail & Related papers (2021-12-17T04:39:33Z) - Identification of Subgroups With Similar Benefits in Off-Policy Policy
Evaluation [60.71312668265873]
We develop a method to balance the need for personalization with confident predictions.
We show that our method can be used to form accurate predictions of heterogeneous treatment effects.
arXiv Detail & Related papers (2021-11-28T23:19:12Z) - Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients [54.98496284653234]
We consider the task of training a policy that maximizes reward while minimizing disclosure of certain sensitive state variables through the actions.
We solve this problem by introducing a regularizer based on the mutual information between the sensitive state and the actions.
We develop a model-based estimator for optimization of privacy-constrained policies.
arXiv Detail & Related papers (2020-12-30T03:22:35Z) - Offline Policy Selection under Uncertainty [113.57441913299868]
We consider offline policy selection as learning preferences over a set of policy prospects given a fixed experience dataset.
Access to the full distribution over one's belief of the policy value enables more flexible selection algorithms under a wider range of downstream evaluation metrics.
We show how BayesDICE may be used to rank policies with respect to any arbitrary downstream policy selection metric.
arXiv Detail & Related papers (2020-12-12T23:09:21Z) - Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic
Policies [80.42316902296832]
We study the estimation of policy value and gradient of a deterministic policy from off-policy data when actions are continuous.
In this setting, standard importance sampling and doubly robust estimators for policy value and gradient fail because the density ratio does not exist.
We propose several new doubly robust estimators based on different kernelization approaches.
arXiv Detail & Related papers (2020-06-06T15:52:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.