End-to-End Learning and Intervention in Games
- URL: http://arxiv.org/abs/2010.13834v1
- Date: Mon, 26 Oct 2020 18:39:32 GMT
- Title: End-to-End Learning and Intervention in Games
- Authors: Jiayang Li, Jing Yu, Yu Marco Nie, Zhaoran Wang
- Abstract summary: We provide a unified framework for learning and intervention in games.
We propose two approaches, respectively based on explicit and implicit differentiation.
The analytical results are validated using several real-world problems.
- Score: 60.41921763076017
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In a social system, the self-interest of agents can be detrimental to the
collective good, sometimes leading to social dilemmas. To resolve such a
conflict, a central designer may intervene by either redesigning the system or
incentivizing the agents to change their behaviors. To be effective, the
designer must anticipate how the agents react to the intervention, which is
dictated by their often unknown payoff functions. Therefore, learning about the
agents is a prerequisite for intervention. In this paper, we provide a unified
framework for learning and intervention in games. We cast the equilibria of
games as individual layers and integrate them into an end-to-end optimization
framework. To enable the backward propagation through the equilibria of games,
we propose two approaches, respectively based on explicit and implicit
differentiation. Specifically, we cast the equilibria as the solutions to
variational inequalities (VIs). The explicit approach unrolls the projection
method for solving VIs, while the implicit approach exploits the sensitivity of
the solutions to VIs. At the core of both approaches is the differentiation
through a projection operator. Moreover, we establish the correctness of both
approaches and identify the conditions under which one approach is more
desirable than the other. The analytical results are validated using several
real-world problems.
Related papers
- Principal-Agent Reinforcement Learning: Orchestrating AI Agents with Contracts [20.8288955218712]
We propose a framework where a principal guides an agent in a Markov Decision Process (MDP) using a series of contracts.
We present and analyze a meta-algorithm that iteratively optimize the policies of the principal and agent.
We then scale our algorithm with deep Q-learning and analyze its convergence in the presence of approximation error.
arXiv Detail & Related papers (2024-07-25T14:28:58Z) - Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment [2.619545850602691]
Recent sociotechnical approaches highlight the need to understand complex misalignment among multiple human and AI agents.
We adapt a computational social science model of human contention to the alignment problem.
Our model quantifies misalignment in large, diverse agent groups with potentially conflicting goals.
arXiv Detail & Related papers (2024-06-06T16:31:22Z) - Optimising Human-AI Collaboration by Learning Convincing Explanations [62.81395661556852]
We propose a method for a collaborative system that remains safe by having a human making decisions.
Ardent enables efficient and effective decision-making by adapting to individual preferences for explanations.
arXiv Detail & Related papers (2023-11-13T16:00:16Z) - Online Decision Mediation [72.80902932543474]
Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior.
In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
arXiv Detail & Related papers (2023-10-28T05:59:43Z) - Formalizing the Problem of Side Effect Regularization [81.97441214404247]
We propose a formal criterion for side effect regularization via the assistance game framework.
In these games, the agent solves a partially observable Markov decision process.
We show that this POMDP is solved by trading off the proxy reward with the agent's ability to achieve a range of future tasks.
arXiv Detail & Related papers (2022-06-23T16:36:13Z) - Inverse Online Learning: Understanding Non-Stationary and Reactionary
Policies [79.60322329952453]
We show how to develop interpretable representations of how agents make decisions.
By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem.
We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them.
Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
arXiv Detail & Related papers (2022-03-14T17:40:42Z) - Distributed Adaptive Learning Under Communication Constraints [54.22472738551687]
This work examines adaptive distributed learning strategies designed to operate under communication constraints.
We consider a network of agents that must solve an online optimization problem from continual observation of streaming data.
arXiv Detail & Related papers (2021-12-03T19:23:48Z) - Fair Incentives for Repeated Engagement [0.46040036610482665]
We study a problem of finding optimal monetary incentive schemes for retention when faced with agents whose participation decisions depend on the incentive they receive.
We show that even in the absence of explicit discrimination, policies may unintentionally discriminate between agents of different types by varying the type composition of the system.
arXiv Detail & Related papers (2021-10-28T04:13:53Z) - On the Fundamental Trade-offs in Learning Invariant Representations [7.868449549351487]
We identify and determine two fundamental trade-offs between utility and semantic dependence induced by the statistical dependencies between the data and its corresponding target and semantic attributes.
We numerically quantify the trade-offs on representative problems and compare to the solutions achieved by baseline representation learning algorithms.
arXiv Detail & Related papers (2021-09-08T01:26:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.