Robust Q-learning

• URL: http://arxiv.org/abs/2003.12427v1
• Date: Fri, 27 Mar 2020 14:10:38 GMT
• Title: Robust Q-learning
• Authors: Ashkan Ertefaie, James R. McKay, David Oslin and Robert L. Strawderman
• Abstract summary: We propose a robust Q-learning approach which allows estimating nuisance parameters using data-adaptive techniques. We study the behavior of our estimators and provide simulation studies that highlight the need for and usefulness of the proposed method.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Q-learning is a regression-based approach that is widely used to formalize the development of an optimal dynamic treatment strategy. Finite dimensional working models are typically used to estimate certain nuisance parameters, and misspecification of these working models can result in residual confounding and/or efficiency loss. We propose a robust Q-learning approach which allows estimating such nuisance parameters using data-adaptive techniques. We study the asymptotic behavior of our estimators and provide simulation studies that highlight the need for and usefulness of the proposed method in practice. We use the data from the "Extending Treatment Effectiveness of Naltrexone" multi-stage randomized trial to illustrate our proposed methods.

Related papers

• Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning [97.2995389188179]
  Recent research has begun to approach large language models (LLMs) unlearning via gradient ascent (GA) Despite their simplicity and efficiency, we suggest that GA-based methods face the propensity towards excessive unlearning. We propose several controlling methods that can regulate the extent of excessive unlearning.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• Towards Learning Stochastic Population Models by Gradient Descent [0.0]
  We show that simultaneous estimation of parameters and structure poses major challenges for optimization procedures. We demonstrate accurate estimation of models but find that enforcing the inference of parsimonious, interpretable models drastically increases the difficulty.
  arXiv  Detail & Related papers  (2024-04-10T14:38:58Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning [50.910152564914405]
  Existing posterior sampling methods for reinforcement learning are limited by being model-based or lack worst-case theoretical guarantees beyond linear MDPs. This paper proposes a new model-free formulation of posterior sampling that applies to more general episodic reinforcement learning problems with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-08-23T12:21:01Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Learning to Refit for Convex Learning Problems [11.464758257681197]
  We propose a framework to learn to estimate optimized model parameters for different training sets using neural networks. We rigorously characterize the power of neural networks to approximate convex problems.
  arXiv  Detail & Related papers  (2021-11-24T15:28:50Z)
• Predictive machine learning for prescriptive applications: a coupled training-validating approach [77.34726150561087]
  We propose a new method for training predictive machine learning models for prescriptive applications. This approach is based on tweaking the validation step in the standard training-validating-testing scheme. Several experiments with synthetic data demonstrate promising results in reducing the prescription costs in both deterministic and real models.
  arXiv  Detail & Related papers  (2021-10-22T15:03:20Z)
• Neural Networks for Parameter Estimation in Intractable Models [0.0]
  We show how to estimate parameters from max-stable processes, where inference is exceptionally challenging. We use data from model simulations as input and train deep neural networks to learn statistical parameters.
  arXiv  Detail & Related papers  (2021-07-29T21:59:48Z)
• A Semiparametric Approach to Interpretable Machine Learning [9.87381939016363]
  Black box models in machine learning have demonstrated excellent predictive performance in complex problems and high-dimensional settings. Their lack of transparency and interpretability restrict the applicability of such models in critical decision-making processes. We propose a novel approach to trading off interpretability and performance in prediction models using ideas from semiparametric statistics.
  arXiv  Detail & Related papers  (2020-06-08T16:38:15Z)
• Model-Augmented Actor-Critic: Backpropagating through Paths [81.86992776864729]
  Current model-based reinforcement learning approaches use the model simply as a learned black-box simulator. We show how to make more effective use of the model by exploiting its differentiability.
  arXiv  Detail & Related papers  (2020-05-16T19:18:10Z)
• Active Learning for Gaussian Process Considering Uncertainties with Application to Shape Control of Composite Fuselage [7.358477502214471]
  We propose two new active learning algorithms for the Gaussian process with uncertainties. We show that the proposed approach can incorporate the impact from uncertainties, and realize better prediction performance. This approach has been applied to improving the predictive modeling for automatic shape control of composite fuselage.
  arXiv  Detail & Related papers  (2020-04-23T02:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.