Contextual Bandits with Budgeted Information Reveal

• URL: http://arxiv.org/abs/2305.18511v3
• Date: Wed, 13 Mar 2024 05:42:44 GMT
• Title: Contextual Bandits with Budgeted Information Reveal
• Authors: Kyra Gan, Esmaeil Keyvanshokooh, Xueqing Liu, Susan Murphy
• Abstract summary: Contextual bandit algorithms are commonly used in digital health to recommend personalized treatments. To ensure the effectiveness of the treatments, patients are often requested to take actions that have no immediate benefit to them. We introduce a novel optimization and learning algorithm to address this problem.
• Score: 3.861395476387163
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Contextual bandit algorithms are commonly used in digital health to recommend personalized treatments. However, to ensure the effectiveness of the treatments, patients are often requested to take actions that have no immediate benefit to them, which we refer to as pro-treatment actions. In practice, clinicians have a limited budget to encourage patients to take these actions and collect additional information. We introduce a novel optimization and learning algorithm to address this problem. This algorithm effectively combines the strengths of two algorithmic approaches in a seamless manner, including 1) an online primal-dual algorithm for deciding the optimal timing to reach out to patients, and 2) a contextual bandit learning algorithm to deliver personalized treatment to the patient. We prove that this algorithm admits a sub-linear regret bound. We illustrate the usefulness of this algorithm on both synthetic and real-world data.

Related papers

• Monitoring Fidelity of Online Reinforcement Learning Algorithms in Clinical Trials [20.944037982124037]
  This paper proposes algorithm fidelity as a critical requirement for deploying online RL algorithms in clinical trials. We present a framework for pre-deployment planning and real-time monitoring to help algorithm developers and clinical researchers ensure algorithm fidelity.
  arXiv  Detail & Related papers  (2024-02-26T20:19:14Z)
• Biomimicry in Radiation Therapy: Optimizing Patient Scheduling for Improved Treatment Outcomes [0.0]
  This study delves into the integration of biomimicry principles within the domain of Radiation Therapy (RT) to optimize patient scheduling. Three bio-inspired algorithms are employed for optimization to tackle the complex online scheduling problem. The results of this study unveil the effectiveness of applied bio-inspired algorithms in optimizing patient scheduling for RT.
  arXiv  Detail & Related papers  (2024-01-16T15:37:23Z)
• Clinical Validation of Single-Chamber Model-Based Algorithms Used to Estimate Respiratory Compliance [2.9511531830032083]
  We establish an open, clinically validated dataset of mechanical lungs and nearly 40,000 breaths from 18 intubated patients. Next, we evaluate 15 different algorithms that use the "single chamber" model of estimating respiratory compliance. In particular, we explore algorithm performance under four different types of patient ventilator asynchrony.
  arXiv  Detail & Related papers  (2021-09-19T07:34:15Z)
• Machine Learning for Online Algorithm Selection under Censored Feedback [71.6879432974126]
  In online algorithm selection (OAS), instances of an algorithmic problem class are presented to an agent one after another, and the agent has to quickly select a presumably best algorithm from a fixed set of candidate algorithms. For decision problems such as satisfiability (SAT), quality typically refers to the algorithm's runtime. In this work, we revisit multi-armed bandit algorithms for OAS and discuss their capability of dealing with the problem. We adapt them towards runtime-oriented losses, allowing for partially censored data while keeping a space- and time-complexity independent of the time horizon.
  arXiv  Detail & Related papers  (2021-09-13T18:10:52Z)
• Deep Algorithm Unrolling for Biomedical Imaging [99.73317152134028]
  In this chapter, we review biomedical applications and breakthroughs via leveraging algorithm unrolling. We trace the origin of algorithm unrolling and provide a comprehensive tutorial on how to unroll iterative algorithms into deep networks. We conclude the chapter by discussing open challenges, and suggesting future research directions.
  arXiv  Detail & Related papers  (2021-08-15T01:06:26Z)
• An Asymptotically Optimal Primal-Dual Incremental Algorithm for Contextual Linear Bandits [129.1029690825929]
  We introduce a novel algorithm improving over the state-of-the-art along multiple dimensions. We establish minimax optimality for any learning horizon in the special case of non-contextual linear bandits.
  arXiv  Detail & Related papers  (2020-10-23T09:12:47Z)
• A black-box adversarial attack for poisoning clustering [78.19784577498031]
  We propose a black-box adversarial attack for crafting adversarial samples to test the robustness of clustering algorithms. We show that our attacks are transferable even against supervised algorithms such as SVMs, random forests, and neural networks.
  arXiv  Detail & Related papers  (2020-09-09T18:19:31Z)
• New Oracle-Efficient Algorithms for Private Synthetic Data Release [52.33506193761153]
  We present three new algorithms for constructing differentially private synthetic data. The algorithms satisfy differential privacy even in the worst case. Compared to the state-of-the-art method High-Dimensional Matrix Mechanism citeMcKennaMHM18, our algorithms provide better accuracy in the large workload.
  arXiv  Detail & Related papers  (2020-07-10T15:46:05Z)
• Run2Survive: A Decision-theoretic Approach to Algorithm Selection based on Survival Analysis [75.64261155172856]
  survival analysis (SA) naturally supports censored data and offers appropriate ways to use such data for learning distributional models of algorithm runtime. We leverage such models as a basis of a sophisticated decision-theoretic approach to algorithm selection, which we dub Run2Survive. In an extensive experimental study with the standard benchmark ASlib, our approach is shown to be highly competitive and in many cases even superior to state-of-the-art AS approaches.
  arXiv  Detail & Related papers  (2020-07-06T15:20:17Z)
• Delay-Adaptive Learning in Generalized Linear Contextual Bandits [18.68458152442088]
  We study the performance of two well-known algorithms adapted to a delayed setting. We describe modifications on how these two algorithms should be adapted to handle delays. Our results contribute to the broad landscape of contextual bandits literature.
  arXiv  Detail & Related papers  (2020-03-11T09:12:44Z)
• Boosting Algorithms for Estimating Optimal Individualized Treatment Rules [4.898659895355356]
  We present nonparametric algorithms for estimating optimal individualized treatment rules. The proposed algorithms are based on the XGBoost algorithm, which is known as one of the most powerful algorithms in the machine learning literature.
  arXiv  Detail & Related papers  (2020-01-31T22:26:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.