Deep Reinforcement Learning in a Monetary Model
- URL: http://arxiv.org/abs/2104.09368v1
- Date: Mon, 19 Apr 2021 14:56:44 GMT
- Title: Deep Reinforcement Learning in a Monetary Model
- Authors: Mingli Chen, Andreas Joseph, Michael Kumhof, Xinlei Pan, Rui Shi, Xuan
Zhou
- Abstract summary: We propose using deep reinforcement learning to solve dynamic general equilibrium models.
Agents are represented by deep artificial neural networks and learn to solve their dynamic optimisation problem.
We find that, contrary to adaptive learning, the artificially intelligent household can solve the model in all policy regimes.
- Score: 5.7742249974375985
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We propose using deep reinforcement learning to solve dynamic stochastic
general equilibrium models. Agents are represented by deep artificial neural
networks and learn to solve their dynamic optimisation problem by interacting
with the model environment, of which they have no a priori knowledge. Deep
reinforcement learning offers a flexible yet principled way to model bounded
rationality within this general class of models. We apply our proposed approach
to a classical model from the adaptive learning literature in macroeconomics
which looks at the interaction of monetary and fiscal policy. We find that,
contrary to adaptive learning, the artificially intelligent household can solve
the model in all policy regimes.
Related papers
- Learning to Steer Markovian Agents under Model Uncertainty [23.603487812521657]
We study how to design additional rewards to steer multi-agent systems towards desired policies.
Motivated by the limitation of existing works, we consider a new category of learning dynamics called emphMarkovian agents
We learn a emphhistory-dependent steering strategy to handle the inherent model uncertainty about the agents' learning dynamics.
arXiv Detail & Related papers (2024-07-14T14:01:38Z) - Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour
with Multi-Agent Reinforcement Learning [4.40301653518681]
Agent-based models (ABMs) have shown promise for modelling various real world phenomena incompatible with traditional equilibrium analysis.
Recent developments in multi-agent reinforcement learning (MARL) offer a way to address this issue from a rationality perspective.
We propose a novel technique for representing heterogeneous processing-constrained agents within a MARL framework.
arXiv Detail & Related papers (2024-02-01T17:21:45Z) - Causal Dynamics Learning for Task-Independent State Abstraction [61.707048209272884]
We introduce Causal Dynamics Learning for Task-Independent State Abstraction (CDL)
CDL learns a theoretically proved causal dynamics model that removes unnecessary dependencies between state variables and the action.
A state abstraction can then be derived from the learned dynamics.
arXiv Detail & Related papers (2022-06-27T17:02:53Z) - Model Generation with Provable Coverability for Offline Reinforcement
Learning [14.333861814143718]
offline optimization with dynamics-aware policy provides a new perspective for policy learning and out-of-distribution generalization.
But due to the limitation under the offline setting, the learned model could not mimic real dynamics well enough to support reliable out-of-distribution exploration.
We propose an algorithm to generate models optimizing their coverage for the real dynamics.
arXiv Detail & Related papers (2022-06-01T08:34:09Z) - Model-based Meta Reinforcement Learning using Graph Structured Surrogate
Models [40.08137765886609]
We show that our model, called a graph structured surrogate model (GSSM), outperforms state-of-the-art methods in predicting environment dynamics.
Our approach is able to obtain high returns, while allowing fast execution during deployment by avoiding test time policy gradient optimization.
arXiv Detail & Related papers (2021-02-16T17:21:55Z) - On the model-based stochastic value gradient for continuous
reinforcement learning [50.085645237597056]
We show that simple model-based agents can outperform state-of-the-art model-free agents in terms of both sample-efficiency and final reward.
Our findings suggest that model-based policy evaluation deserves closer attention.
arXiv Detail & Related papers (2020-08-28T17:58:29Z) - Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy.
We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space.
We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
arXiv Detail & Related papers (2020-07-14T16:42:59Z) - Context-aware Dynamics Model for Generalization in Model-Based
Reinforcement Learning [124.9856253431878]
We decompose the task of learning a global dynamics model into two stages: (a) learning a context latent vector that captures the local dynamics, then (b) predicting the next state conditioned on it.
In order to encode dynamics-specific information into the context latent vector, we introduce a novel loss function that encourages the context latent vector to be useful for predicting both forward and backward dynamics.
The proposed method achieves superior generalization ability across various simulated robotics and control tasks, compared to existing RL schemes.
arXiv Detail & Related papers (2020-05-14T08:10:54Z) - Learning Stable Deep Dynamics Models [91.90131512825504]
We propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space.
We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics.
arXiv Detail & Related papers (2020-01-17T00:04:45Z) - Information Theoretic Model Predictive Q-Learning [64.74041985237105]
We present a novel theoretical connection between information theoretic MPC and entropy regularized RL.
We develop a Q-learning algorithm that can leverage biased models.
arXiv Detail & Related papers (2019-12-31T00:29:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.