Policy Design in Long-Run Welfare Dynamics

• URL: http://arxiv.org/abs/2503.00632v1
• Date: Sat, 01 Mar 2025 21:50:57 GMT
• Title: Policy Design in Long-Run Welfare Dynamics
• Authors: Jiduan Wu, Rediet Abebe, Moritz Hardt, Ana-Andreea Stoica,
• Abstract summary: We analyze the long-term dynamics of two prominent policy frameworks: Rawlsian policies, which prioritize those with the greatest need, and utilitarian policies, which maximize immediate welfare gains.<n>We prove that interventions following Rawlsian policies can outperform utilitarian policies in the long run, even when the latter dominate in the short run.<n>Our results underscore the necessity of considering long-term horizons in designing and evaluating welfare policies.
• Score: 21.242427640040717
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Improving social welfare is a complex challenge requiring policymakers to optimize objectives across multiple time horizons. Evaluating the impact of such policies presents a fundamental challenge, as those that appear suboptimal in the short run may yield significant long-term benefits. We tackle this challenge by analyzing the long-term dynamics of two prominent policy frameworks: Rawlsian policies, which prioritize those with the greatest need, and utilitarian policies, which maximize immediate welfare gains. Conventional wisdom suggests these policies are at odds, as Rawlsian policies are assumed to come at the cost of reducing the average social welfare, which their utilitarian counterparts directly optimize. We challenge this assumption by analyzing these policies in a sequential decision-making framework where individuals' welfare levels stochastically decay over time, and policymakers can intervene to prevent this decay. Under reasonable assumptions, we prove that interventions following Rawlsian policies can outperform utilitarian policies in the long run, even when the latter dominate in the short run. We characterize the exact conditions under which Rawlsian policies can outperform utilitarian policies. We further illustrate our theoretical findings using simulations, which highlight the risks of evaluating policies based solely on their short-term effects. Our results underscore the necessity of considering long-term horizons in designing and evaluating welfare policies; the true efficacy of even well-established policies may only emerge over time.

Related papers

• Residual Policy Gradient: A Reward View of KL-regularized Objective [48.39829592175419]
  Reinforcement Learning and Imitation Learning have achieved widespread success in many domains but remain constrained during real-world deployment. Policy customization has been introduced, aiming to adapt a prior policy while preserving its inherent properties and meeting new task-specific requirements. A principled approach to policy customization is Residual Q-Learning (RQL), which formulates the problem as a Markov Decision Process (MDP) and derives a family of value-based learning algorithms. We introduce Residual Policy Gradient (RPG), which extends RQL to policy gradient methods, allowing policy customization in gradient-based RL settings.
  arXiv  Detail & Related papers  (2025-03-14T02:30:13Z)
• On the Value of Myopic Behavior in Policy Reuse [67.37788288093299]
  Leveraging learned strategies in unfamiliar scenarios is fundamental to human intelligence. In this work, we present a framework called Selective Myopic bEhavior Control(SMEC) SMEC adaptively aggregates the sharable short-term behaviors of prior policies and the long-term behaviors of the task policy, leading to coordinated decisions.
  arXiv  Detail & Related papers  (2023-05-28T03:59:37Z)
• Policy Learning with Competing Agents [2.972870935419738]
  Decision makers often aim to learn a treatment assignment policy under a capacity constraint on the number of agents that they can treat. In this paper, we study capacity-constrained treatment assignment in the presence of such interference.
  arXiv  Detail & Related papers  (2022-04-04T23:15:00Z)
• An Alternate Policy Gradient Estimator for Softmax Policies [36.48028448548086]
  We propose a novel policy gradient estimator for softmax policies. Our analysis and experiments, conducted on bandits and classical MDP benchmarking tasks, show that our estimator is more robust to policy saturation.
  arXiv  Detail & Related papers  (2021-12-22T02:01:19Z)
• Building a Foundation for Data-Driven, Interpretable, and Robust Policy Design using the AI Economist [67.08543240320756]
  We show that the AI Economist framework enables effective, flexible, and interpretable policy design using two-level reinforcement learning and data-driven simulations. We find that log-linear policies trained using RL significantly improve social welfare, based on both public health and economic outcomes, compared to past outcomes.
  arXiv  Detail & Related papers  (2021-08-06T01:30:41Z)
• Offline Policy Selection under Uncertainty [113.57441913299868]
  We consider offline policy selection as learning preferences over a set of policy prospects given a fixed experience dataset. Access to the full distribution over one's belief of the policy value enables more flexible selection algorithms under a wider range of downstream evaluation metrics. We show how BayesDICE may be used to rank policies with respect to any arbitrary downstream policy selection metric.
  arXiv  Detail & Related papers  (2020-12-12T23:09:21Z)
• Targeting for long-term outcomes [1.7205106391379026]
  Decision makers often want to target interventions so as to maximize an outcome that is observed only in the long-term. Here we build on the statistical surrogacy and policy learning literatures to impute the missing long-term outcomes. We apply our approach in two large-scale proactive churn management experiments at The Boston Globe.
  arXiv  Detail & Related papers  (2020-10-29T18:31:17Z)
• Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies [80.42316902296832]
  We study the estimation of policy value and gradient of a deterministic policy from off-policy data when actions are continuous. In this setting, standard importance sampling and doubly robust estimators for policy value and gradient fail because the density ratio does not exist. We propose several new doubly robust estimators based on different kernelization approaches.
  arXiv  Detail & Related papers  (2020-06-06T15:52:05Z)
• Efficient Evaluation of Natural Stochastic Policies in Offline Reinforcement Learning [80.42316902296832]
  We study the efficient off-policy evaluation of natural policies, which are defined in terms of deviations from the behavior policy. This is a departure from the literature on off-policy evaluation where most work consider the evaluation of explicitly specified policies.
  arXiv  Detail & Related papers  (2020-06-06T15:08:24Z)
• Stable Policy Optimization via Off-Policy Divergence Regularization [50.98542111236381]
  Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are among the most successful policy gradient approaches in deep reinforcement learning (RL) We propose a new algorithm which stabilizes the policy improvement through a proximity term that constrains the discounted state-action visitation distribution induced by consecutive policies to be close to one another. Our proposed method can have a beneficial effect on stability and improve final performance in benchmark high-dimensional control tasks.
  arXiv  Detail & Related papers  (2020-03-09T13:05:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.