Learning with Limited Shared Information in Multi-agent Multi-armed Bandit

• URL: http://arxiv.org/abs/2502.15338v1
• Date: Fri, 21 Feb 2025 09:42:09 GMT
• Title: Learning with Limited Shared Information in Multi-agent Multi-armed Bandit
• Authors: Junning Shao, Siwei Wang, Zhixuan Fang,
• Abstract summary: Multi-agent multi-armed bandit (MAMAB) is a classic collaborative learning model and has gained much attention in recent years.<n>We propose a novel limited shared information multi-agent multi-armed bandit (LSI-MAMAB) model in which each agent only shares the information that she is willing to share.
• Score: 28.82167431329527
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent multi-armed bandit (MAMAB) is a classic collaborative learning model and has gained much attention in recent years. However, existing studies do not consider the case where an agent may refuse to share all her information with others, e.g., when some of the data contains personal privacy. In this paper, we propose a novel limited shared information multi-agent multi-armed bandit (LSI-MAMAB) model in which each agent only shares the information that she is willing to share, and propose the Balanced-ETC algorithm to help multiple agents collaborate efficiently with limited shared information. Our analysis shows that Balanced-ETC is asymptotically optimal and its average regret (on each agent) approaches a constant when there are sufficient agents involved. Moreover, to encourage agents to participate in this collaborative learning, an incentive mechanism is proposed to make sure each agent can benefit from the collaboration system. Finally, we present experimental results to validate our theoretical results.

Related papers

• COMMA: A Communicative Multimodal Multi-Agent Benchmark [7.831385481814481]
  We introduce a novel benchmark designed to evaluate the collaborative performance of multimodal multi-agent systems through language communication. Our findings reveal surprising weaknesses in state-of-the-art models, including proprietary models like GPT-4o.
  arXiv  Detail & Related papers  (2024-10-10T02:49:47Z)
• COMBO: Compositional World Models for Embodied Multi-Agent Cooperation [64.27636858152522]
  Decentralized agents must cooperate given only partial egocentric views of the world. We train generative models to estimate the overall world state given partial egocentric observations. We learn a compositional world model for multi-agent cooperation by factorizing the naturally composable joint actions of multiple agents.
  arXiv  Detail & Related papers  (2024-04-16T17:59:11Z)
• Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination [16.74629849552254]
  We propose a model-based consensus mechanism to explicitly coordinate multiple agents. The proposed Multi-agent Goal Imagination (MAGI) framework guides agents to reach consensus with an Imagined common goal. We show that such efficient consensus mechanism can guide all agents cooperatively reaching valuable future states.
  arXiv  Detail & Related papers  (2024-03-05T18:07:34Z)
• Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey [48.77342627610471]
  Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks. It is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting.
  arXiv  Detail & Related papers  (2024-01-10T05:07:42Z)
• Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning [9.25057318925143]
  We present a novel multi-agent RL approach, in which agents share with other agents a limited number of transitions they observe during training. We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms.
  arXiv  Detail & Related papers  (2023-11-01T21:35:32Z)
• Deep Multi-Agent Reinforcement Learning for Decentralized Active Hypothesis Testing [11.639503711252663]
  We tackle the multi-agent active hypothesis testing (AHT) problem by introducing a novel algorithm rooted in the framework of deep multi-agent reinforcement learning. We present a comprehensive set of experimental results that effectively showcase the agents' ability to learn collaborative strategies and enhance performance.
  arXiv  Detail & Related papers  (2023-09-14T01:18:04Z)
• Pure Exploration under Mediators' Feedback [63.56002444692792]
  Multi-armed bandits are a sequential-decision-making framework, where, at each interaction step, the learner selects an arm and observes a reward. We consider the scenario in which the learner has access to a set of mediators, each of which selects the arms on the agent's behalf according to a and possibly unknown policy. We propose a sequential decision-making strategy for discovering the best arm under the assumption that the mediators' policies are known to the learner.
  arXiv  Detail & Related papers  (2023-08-29T18:18:21Z)
• Multi-agent Deep Covering Skill Discovery [50.812414209206054]
  We propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space. Also, we propose a novel framework to adopt the multi-agent options in the MARL process. We show that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options.
  arXiv  Detail & Related papers  (2022-10-07T00:40:59Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Celebrating Diversity in Shared Multi-Agent Reinforcement Learning [20.901606233349177]
  Deep multi-agent reinforcement learning has shown the promise to solve complex cooperative tasks. In this paper, we aim to introduce diversity in both optimization and representation of shared multi-agent reinforcement learning. Our method achieves state-of-the-art performance on Google Research Football and super hard StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-06-04T00:55:03Z)
• Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic [54.2180984002807]
  Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems. We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works.
  arXiv  Detail & Related papers  (2020-02-24T20:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.