Provably Efficient Policy Gradient Methods for Two-Player Zero-Sum Markov Games

• URL: http://arxiv.org/abs/2102.08903v1
• Date: Wed, 17 Feb 2021 17:49:57 GMT
• Title: Provably Efficient Policy Gradient Methods for Two-Player Zero-Sum Markov Games
• Authors: Yulai Zhao, Yuandong Tian, Jason D. Lee, Simon S. Du
• Abstract summary: This paper studies natural extensions of Natural Policy Gradient algorithm for solving two-player zero-sum games. We thoroughly characterize the algorithms' performance in terms of the number of samples, number of iterations, concentrability coefficients, and approximation error.
• Score: 95.70078702838654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Policy gradient methods are widely used in solving two-player zero-sum games to achieve superhuman performance in practice. However, it remains elusive when they can provably find a near-optimal solution and how many samples and iterations are needed. The current paper studies natural extensions of Natural Policy Gradient algorithm for solving two-player zero-sum games where function approximation is used for generalization across states. We thoroughly characterize the algorithms' performance in terms of the number of samples, number of iterations, concentrability coefficients, and approximation error. To our knowledge, this is the first quantitative analysis of policy gradient methods with function approximation for two-player zero-sum Markov games.

Related papers

Err

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.