Learning with Limited Shared Information in Multi-agent Multi-armed Bandit

• URL: http://arxiv.org/abs/2502.15338v1
• Date: Fri, 21 Feb 2025 09:42:09 GMT
• Title: Learning with Limited Shared Information in Multi-agent Multi-armed Bandit
• Abstract summary: Multi-agent multi-armed bandit (MAMAB) is a classic collaborative learning model and has gained much attention in recent years.<n>We propose a novel limited shared information multi-agent multi-armed bandit (LSI-MAMAB) model in which each agent only shares the information that she is willing to share.
• Score: 28.82167431329527
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent multi-armed bandit (MAMAB) is a classic collaborative learning model and has gained much attention in recent years. However, existing studies do not consider the case where an agent may refuse to share all her information with others, e.g., when some of the data contains personal privacy. In this paper, we propose a novel limited shared information multi-agent multi-armed bandit (LSI-MAMAB) model in which each agent only shares the information that she is willing to share, and propose the Balanced-ETC algorithm to help multiple agents collaborate efficiently with limited shared information. Our analysis shows that Balanced-ETC is asymptotically optimal and its average regret (on each agent) approaches a constant when there are sufficient agents involved. Moreover, to encourage agents to participate in this collaborative learning, an incentive mechanism is proposed to make sure each agent can benefit from the collaboration system. Finally, we present experimental results to validate our theoretical results.

Related papers

• Collaborating in Multi-Armed Bandits with Strategic Agents [37.473632043513014]
  We study collaborative learning in multi-agent Bayesian bandit problems, where strategic agents collectively solve the same bandit instance.<n>While multiple agents can accelerate learning by sharing information, strategic agents might prefer to free-ride and avoid exploration.<n>We propose textttCAOS, a mechanism that sustains collaboration as a Nash equilibrium while achieving strong regret guarantees.
  arXiv  Detail & Related papers  (2026-05-13T08:10:36Z)
• Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning [112.16686518063456]
  We introduce textbfMulti-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time.<n>MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making.<n>Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67% over a multi-agent baseline, and by 8.67% over comparable single-agent baselines
  arXiv  Detail & Related papers  (2026-01-14T17:57:43Z)
• Can an Individual Manipulate the Collective Decisions of Multi-Agents? [53.01767232004823]
  M-Spoiler is a framework that simulates agent interactions within a multi-agent system to generate adversarial samples.<n>M-Spoiler introduces a stubborn agent that actively aids in optimizing adversarial samples.<n>Our findings confirm the risks posed by the knowledge of an individual agent in multi-agent systems.
  arXiv  Detail & Related papers  (2025-09-20T01:54:20Z)
• COMMA: A Communicative Multimodal Multi-Agent Benchmark [7.831385481814481]
  We introduce a novel benchmark designed to evaluate the collaborative performance of multimodal multi-agent systems through language communication. Our findings reveal surprising weaknesses in state-of-the-art models, including proprietary models like GPT-4o.
  arXiv  Detail & Related papers  (2024-10-10T02:49:47Z)
• COMBO: Compositional World Models for Embodied Multi-Agent Cooperation [64.27636858152522]
  Decentralized agents must cooperate given only partial egocentric views of the world. We train generative models to estimate the overall world state given partial egocentric observations. We learn a compositional world model for multi-agent cooperation by factorizing the naturally composable joint actions of multiple agents.
  arXiv  Detail & Related papers  (2024-04-16T17:59:11Z)
• Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination [16.74629849552254]
  We propose a model-based consensus mechanism to explicitly coordinate multiple agents. The proposed Multi-agent Goal Imagination (MAGI) framework guides agents to reach consensus with an Imagined common goal. We show that such efficient consensus mechanism can guide all agents cooperatively reaching valuable future states.
  arXiv  Detail & Related papers  (2024-03-05T18:07:34Z)
• Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey [48.77342627610471]
  Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks. It is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting.
  arXiv  Detail & Related papers  (2024-01-10T05:07:42Z)
• Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning [9.25057318925143]
  We present a novel multi-agent RL approach, in which agents share with other agents a limited number of transitions they observe during training. We show that our approach outperforms baseline no-sharing decentralized training and state-of-the art multi-agent RL algorithms.
  arXiv  Detail & Related papers  (2023-11-01T21:35:32Z)
• Deep Multi-Agent Reinforcement Learning for Decentralized Active Hypothesis Testing [11.639503711252663]
  We tackle the multi-agent active hypothesis testing (AHT) problem by introducing a novel algorithm rooted in the framework of deep multi-agent reinforcement learning. We present a comprehensive set of experimental results that effectively showcase the agents' ability to learn collaborative strategies and enhance performance.
  arXiv  Detail & Related papers  (2023-09-14T01:18:04Z)
• Pure Exploration under Mediators' Feedback [63.56002444692792]
  Multi-armed bandits are a sequential-decision-making framework, where, at each interaction step, the learner selects an arm and observes a reward. We consider the scenario in which the learner has access to a set of mediators, each of which selects the arms on the agent's behalf according to a and possibly unknown policy. We propose a sequential decision-making strategy for discovering the best arm under the assumption that the mediators' policies are known to the learner.
  arXiv  Detail & Related papers  (2023-08-29T18:18:21Z)
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors [93.38830440346783]
  We propose a multi-agent framework framework that can collaboratively adjust its composition as a greater-than-the-sum-of-its-parts system. Our experiments demonstrate that framework framework can effectively deploy multi-agent groups that outperform a single agent. In view of these behaviors, we discuss some possible strategies to leverage positive ones and mitigate negative ones for improving the collaborative potential of multi-agent groups.
  arXiv  Detail & Related papers  (2023-08-21T16:47:11Z)
• Multi-agent Deep Covering Skill Discovery [50.812414209206054]
  We propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space. Also, we propose a novel framework to adopt the multi-agent options in the MARL process. We show that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options.
  arXiv  Detail & Related papers  (2022-10-07T00:40:59Z)
• Partner-Aware Algorithms in Decentralized Cooperative Bandit Teams [14.215359943041369]
  We propose and analyze a decentralized Multi-Armed Bandit (MAB) problem with coupled rewards as an abstraction of more general multi-agent collaboration. We propose a Partner-Aware strategy for joint sequential decision-making that extends the well-known single-agent Upper Confidence Bound algorithm. Our results show that the proposed partner-aware strategy outperforms other known methods, and our human subject studies suggest humans prefer to collaborate with AI agents implementing our partner-aware strategy.
  arXiv  Detail & Related papers  (2021-10-02T08:17:30Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Celebrating Diversity in Shared Multi-Agent Reinforcement Learning [20.901606233349177]
  Deep multi-agent reinforcement learning has shown the promise to solve complex cooperative tasks. In this paper, we aim to introduce diversity in both optimization and representation of shared multi-agent reinforcement learning. Our method achieves state-of-the-art performance on Google Research Football and super hard StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-06-04T00:55:03Z)
• Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic [54.2180984002807]
  Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems. We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works.
  arXiv  Detail & Related papers  (2020-02-24T20:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.