Exponential Family Model-Based Reinforcement Learning via Score Matching

• URL: http://arxiv.org/abs/2112.14195v1
• Date: Tue, 28 Dec 2021 15:51:07 GMT
• Title: Exponential Family Model-Based Reinforcement Learning via Score Matching
• Authors: Gene Li, Junbo Li, Nathan Srebro, Zhaoran Wang, Zhuoran Yang
• Abstract summary: We propose an optimistic model-based algorithm, dubbed SMRL, for finitehorizon episodic reinforcement learning (RL) SMRL uses score matching, an unnormalized density estimation technique that enables efficient estimation of the model parameter by ridge regression.
• Score: 97.31477125728844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose an optimistic model-based algorithm, dubbed SMRL, for finite-horizon episodic reinforcement learning (RL) when the transition model is specified by exponential family distributions with $d$ parameters and the reward is bounded and known. SMRL uses score matching, an unnormalized density estimation technique that enables efficient estimation of the model parameter by ridge regression. Under standard regularity assumptions, SMRL achieves $\tilde O(d\sqrt{H^3T})$ online regret, where $H$ is the length of each episode and $T$ is the total number of interactions (ignoring polynomial dependence on structural scale parameters).

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.