Information Design in Multi-Agent Reinforcement Learning
- URL: http://arxiv.org/abs/2305.06807v2
- Date: Sun, 29 Oct 2023 12:30:15 GMT
- Title: Information Design in Multi-Agent Reinforcement Learning
- Authors: Yue Lin, Wenhao Li, Hongyuan Zha, Baoxiang Wang
- Abstract summary: Reinforcement learning (RL) is inspired by the way human infants and animals learn from the environment.
Research in computational economics distills two ways to influence others directly: by providing tangible goods (mechanism design) and by providing information (information design)
- Score: 61.140924904755266
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reinforcement learning (RL) is inspired by the way human infants and animals
learn from the environment. The setting is somewhat idealized because, in
actual tasks, other agents in the environment have their own goals and behave
adaptively to the ego agent. To thrive in those environments, the agent needs
to influence other agents so their actions become more helpful and less
harmful. Research in computational economics distills two ways to influence
others directly: by providing tangible goods (mechanism design) and by
providing information (information design). This work investigates information
design problems for a group of RL agents. The main challenges are two-fold. One
is the information provided will immediately affect the transition of the agent
trajectories, which introduces additional non-stationarity. The other is the
information can be ignored, so the sender must provide information that the
receiver is willing to respect. We formulate the Markov signaling game, and
develop the notions of signaling gradient and the extended obedience
constraints that address these challenges. Our algorithm is efficient on
various mixed-motive tasks and provides further insights into computational
economics. Our code is publicly available at
https://github.com/YueLin301/InformationDesignMARL.
Related papers
- Semifactual Explanations for Reinforcement Learning [1.5320737596132754]
Reinforcement Learning (RL) is a learning paradigm in which the agent learns from its environment through trial and error.
Deep reinforcement learning (DRL) algorithms represent the agent's policies using neural networks, making their decisions difficult to interpret.
Explaining the behaviour of DRL agents is necessary to advance user trust, increase engagement, and facilitate integration with real-life tasks.
arXiv Detail & Related papers (2024-09-09T08:37:47Z) - AgentGym: Evolving Large Language Model-based Agents across Diverse Environments [116.97648507802926]
Large language models (LLMs) are considered a promising foundation to build such agents.
We take the first step towards building generally-capable LLM-based agents with self-evolution ability.
We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration.
arXiv Detail & Related papers (2024-06-06T15:15:41Z) - Knowledge Boundary and Persona Dynamic Shape A Better Social Media Agent [69.12885360755408]
We construct a social media agent based on personalized knowledge and dynamic persona information.
For personalized knowledge, we add external knowledge sources and match them with the persona information of agents, thereby giving the agent personalized world knowledge.
For dynamic persona information, we use current action information to internally retrieve the persona information of the agent, thereby reducing the interference of diverse persona information on the current action.
arXiv Detail & Related papers (2024-03-28T10:01:23Z) - Ignorance is Bliss: Robust Control via Information Gating [60.17644038829572]
Informational parsimony provides a useful inductive bias for learning representations that achieve better generalization by being robust to noise and spurious correlations.
We propose textitinformation gating as a way to learn parsimonious representations that identify the minimal information required for a task.
arXiv Detail & Related papers (2023-03-10T18:31:50Z) - GANterfactual-RL: Understanding Reinforcement Learning Agents'
Strategies through Visual Counterfactual Explanations [0.7874708385247353]
We propose a novel but simple method to generate counterfactual explanations for RL agents.
Our method is fully model-agnostic and we demonstrate that it outperforms the only previous method in several computational metrics.
arXiv Detail & Related papers (2023-02-24T15:29:43Z) - Denoised MDPs: Learning World Models Better Than the World Itself [94.74665254213588]
This work categorizes information out in the wild into four types based on controllability and relation with reward, and formulates useful information as that which is both controllable and reward-relevant.
Experiments on variants of DeepMind Control Suite and RoboDesk demonstrate superior performance of our denoised world model over using raw observations alone.
arXiv Detail & Related papers (2022-06-30T17:59:49Z) - Retrieval-Augmented Reinforcement Learning [63.32076191982944]
We train a network to map a dataset of past experiences to optimal behavior.
The retrieval process is trained to retrieve information from the dataset that may be useful in the current context.
We show that retrieval-augmented R2D2 learns significantly faster than the baseline R2D2 agent and achieves higher scores.
arXiv Detail & Related papers (2022-02-17T02:44:05Z) - Reinforcement Learning, Bit by Bit [27.66567077899924]
Reinforcement learning agents have demonstrated remarkable achievements in simulated environments.
Data efficiency poses an impediment to carrying this success over to real environments.
We discuss concepts and regret analysis that together offer principled guidance.
arXiv Detail & Related papers (2021-03-06T06:37:46Z) - On the Equilibrium Elicitation of Markov Games Through Information
Design [32.37168850559519]
We study how the craft of payoffrelevant environmental signals solely can influence the behaviors of intelligent agents.
An obedient principle is established which states that it is without loss of generality to focus on the direct information design.
A new framework for information design is proposed based on an approach of maximizing the optimal slack variables.
arXiv Detail & Related papers (2021-02-14T13:30:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.