Can Reinforcement Learning support policy makers? A preliminary study with Integrated Assessment Models

• URL: http://arxiv.org/abs/2312.06527v1
• Date: Mon, 11 Dec 2023 17:04:30 GMT
• Title: Can Reinforcement Learning support policy makers? A preliminary study with Integrated Assessment Models
• Authors: Theodore Wolf and Nantas Nardelli and John Shawe-Taylor and Maria Perez-Ortiz
• Abstract summary: Integrated Assessment Models (IAMs) attempt to link main features of society and economy with the biosphere into one modelling framework. This paper empirically demonstrates that modern Reinforcement Learning can be used to probe IAMs and explore the space of solutions in a more principled manner.
• Score: 7.1307809008103735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Governments around the world aspire to ground decision-making on evidence. Many of the foundations of policy making - e.g. sensing patterns that relate to societal needs, developing evidence-based programs, forecasting potential outcomes of policy changes, and monitoring effectiveness of policy programs - have the potential to benefit from the use of large-scale datasets or simulations together with intelligent algorithms. These could, if designed and deployed in a way that is well grounded on scientific evidence, enable a more comprehensive, faster, and rigorous approach to policy making. Integrated Assessment Models (IAM) is a broad umbrella covering scientific models that attempt to link main features of society and economy with the biosphere into one modelling framework. At present, these systems are probed by policy makers and advisory groups in a hypothesis-driven manner. In this paper, we empirically demonstrate that modern Reinforcement Learning can be used to probe IAMs and explore the space of solutions in a more principled manner. While the implication of our results are modest since the environment is simplistic, we believe that this is a stepping stone towards more ambitious use cases, which could allow for effective exploration of policies and understanding of their consequences and limitations.

Related papers

• Large Legislative Models: Towards Efficient AI Policymaking in Economic Simulations [4.153442346657272]
  AI policymaking holds the potential to surpass human performance through the ability to process data quickly at scale. Existing RL-based methods exhibit sample inefficiency, and are further limited by an inability to flexibly incorporate nuanced information into their decision-making processes. We propose a novel method in which we instead utilize pre-trained Large Language Models (LLMs), as sample-efficient policymakers.
  arXiv  Detail & Related papers  (2024-10-10T20:04:58Z)
• Building Understandable Messaging for Policy and Evidence Review (BUMPER) with AI [0.3495246564946556]
  We introduce a framework for the use of large language models (LLMs) in Building Understandable Messaging for Policy and Evidence Review (BUMPER) LLMs are capable of providing interfaces for understanding and synthesizing large databases of diverse media. We argue that this framework can facilitate accessibility of and confidence in scientific evidence for policymakers.
  arXiv  Detail & Related papers  (2024-06-27T05:03:03Z)
• Learning Optimal Deterministic Policies with Stochastic Policy Gradients [62.81324245896716]
  Policy gradient (PG) methods are successful approaches to deal with continuous reinforcement learning (RL) problems. In common practice, convergence (hyper)policies are learned only to deploy their deterministic version. We show how to tune the exploration level used for learning to optimize the trade-off between the sample complexity and the performance of the deployed deterministic policy.
  arXiv  Detail & Related papers  (2024-05-03T16:45:15Z)
• Counterfactual Explanation Policies in RL [3.674863913115432]
  COUNTERPOL is the first framework to analyze Reinforcement Learning policies using counterfactual explanations. We establish a theoretical connection between Counterpol and widely used trust region-based policy optimization methods in RL.
  arXiv  Detail & Related papers  (2023-07-25T01:14:56Z)
• Dual policy as self-model for planning [71.73710074424511]
  We refer to the model used to simulate one's decisions as the agent's self-model. Inspired by current reinforcement learning approaches and neuroscience, we explore the benefits and limitations of using a distilled policy network as the self-model.
  arXiv  Detail & Related papers  (2023-06-07T13:58:45Z)
• Representation-Driven Reinforcement Learning [57.44609759155611]
  We present a representation-driven framework for reinforcement learning. By representing policies as estimates of their expected values, we leverage techniques from contextual bandits to guide exploration and exploitation. We demonstrate the effectiveness of this framework through its application to evolutionary and policy gradient-based approaches.
  arXiv  Detail & Related papers  (2023-05-31T14:59:12Z)
• Verified Probabilistic Policies for Deep Reinforcement Learning [6.85316573653194]
  We tackle the problem of verifying probabilistic policies for deep reinforcement learning. We propose an abstraction approach, based on interval Markov decision processes, that yields guarantees on a policy's execution. We present techniques to build and solve these models using abstract interpretation, mixed-integer linear programming, entropy-based refinement and probabilistic model checking.
  arXiv  Detail & Related papers  (2022-01-10T23:55:04Z)
• Constructing a Good Behavior Basis for Transfer using Generalized Policy Updates [63.58053355357644]
  We study the problem of learning a good set of policies, so that when combined together, they can solve a wide variety of unseen reinforcement learning tasks. We show theoretically that having access to a specific set of diverse policies, which we call a set of independent policies, can allow for instantaneously achieving high-level performance.
  arXiv  Detail & Related papers  (2021-12-30T12:20:46Z)
• Building a Foundation for Data-Driven, Interpretable, and Robust Policy Design using the AI Economist [67.08543240320756]
  We show that the AI Economist framework enables effective, flexible, and interpretable policy design using two-level reinforcement learning and data-driven simulations. We find that log-linear policies trained using RL significantly improve social welfare, based on both public health and economic outcomes, compared to past outcomes.
  arXiv  Detail & Related papers  (2021-08-06T01:30:41Z)
• Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning [75.56839075060819]
  Traditional robotic approaches rely on an accurate model of the environment, a detailed description of how to perform the task, and a robust perception system to keep track of the current state. reinforcement learning approaches can operate directly from raw sensory inputs with only a reward signal to describe the task, but are extremely sample-inefficient and brittle. In this work, we combine the strengths of model-based methods with the flexibility of learning-based methods to obtain a general method that is able to overcome inaccuracies in the robotics perception/actuation pipeline.
  arXiv  Detail & Related papers  (2020-05-21T19:47:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.