Constructive Conflict-Driven Multi-Agent Reinforcement Learning for Strategic Diversity

• URL: http://arxiv.org/abs/2509.14276v2
• Date: Fri, 26 Sep 2025 02:28:36 GMT
• Title: Constructive Conflict-Driven Multi-Agent Reinforcement Learning for Strategic Diversity
• Authors: Yuxiang Mai, Qiyue Yin, Wancheng Ni, Pei Xu, Kaiqi Huang,
• Abstract summary: We propose Competitive Diversity through Constructive Conflict (CoDiCon), a novel approach that incorporates competitive incentives into cooperative scenarios.<n>A central intrinsic reward module generates and distributes varying reward values to agents, ensuring an effective balance between competition and cooperation.<n> Experimental results demonstrate that CoDiCon achieves superior performance, with competitive intrinsic rewards effectively promoting diverse and adaptive strategies.
• Score: 27.335624335134018
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, diversity has emerged as a useful mechanism to enhance the efficiency of multi-agent reinforcement learning (MARL). However, existing methods predominantly focus on designing policies based on individual agent characteristics, often neglecting the interplay and mutual influence among agents during policy formation. To address this gap, we propose Competitive Diversity through Constructive Conflict (CoDiCon), a novel approach that incorporates competitive incentives into cooperative scenarios to encourage policy exchange and foster strategic diversity among agents. Drawing inspiration from sociological research, which highlights the benefits of moderate competition and constructive conflict in group decision-making, we design an intrinsic reward mechanism using ranking features to introduce competitive motivations. A centralized intrinsic reward module generates and distributes varying reward values to agents, ensuring an effective balance between competition and cooperation. By optimizing the parameterized centralized reward module to maximize environmental rewards, we reformulate the constrained bilevel optimization problem to align with the original task objectives. We evaluate our algorithm against state-of-the-art methods in the SMAC and GRF environments. Experimental results demonstrate that CoDiCon achieves superior performance, with competitive intrinsic rewards effectively promoting diverse and adaptive strategies among cooperative agents.

Related papers

• Heterogeneous Agent Collaborative Reinforcement Learning [52.99813668995983]
  Heterogeneous Agent Collaborative Reinforcement Learning (HACRL)<n>Building on this paradigm, we propose HACPO, a collaborative RL algorithm that enables principled rollout sharing to maximize sample utilization and cross-agent knowledge transfer.<n>Experiments across diverse heterogeneous model combinations and reasoning benchmarks show that HACPO consistently improves all participating agents, outperforming GSPO by an average of 3.3% while using only half the rollout cost.
  arXiv  Detail & Related papers  (2026-03-03T05:09:49Z)
• Enhancing Multi-Agent Collaboration with Attention-Based Actor-Critic Policies [0.0]
  Team-Attention-Actor-Critic (TAAC) is a learning algorithm designed to enhance multi-agent collaboration in cooperative environments.<n>We evaluate TAAC in a simulated soccer environment against benchmark algorithms.
  arXiv  Detail & Related papers  (2025-07-30T15:48:38Z)
• Achieving Collective Welfare in Multi-Agent Reinforcement Learning via Suggestion Sharing [12.167248367980449]
  Conflict between self-interest and collective well-being often obstructs efforts to achieve shared welfare.<n>We propose a novel multi-agent reinforcement learning (MARL) method to address this issue.<n>Unlike traditional cooperative MARL solutions that involve sharing rewards, values, and policies, we propose a novel MARL approach where agents exchange action suggestions.
  arXiv  Detail & Related papers  (2024-12-16T19:44:44Z)
• Role Play: Learning Adaptive Role-Specific Strategies in Multi-Agent Interactions [8.96091816092671]
  We propose a novel framework called emphRole Play (RP) RP employs role embeddings to transform the challenge of policy diversity into a more manageable diversity of roles. It trains a common policy with role embedding observations and employs a role predictor to estimate the joint role embeddings of other agents, helping the learning agent adapt to its assigned role.
  arXiv  Detail & Related papers  (2024-11-02T07:25:48Z)
• Learning to Balance Altruism and Self-interest Based on Empathy in Mixed-Motive Games [47.8980880888222]
  Multi-agent scenarios often involve mixed motives, demanding altruistic agents capable of self-protection against potential exploitation.<n>We propose LASE Learning to balance Altruism and Self-interest based on Empathy.<n>LASE allocates a portion of its rewards to co-players as gifts, with this allocation adapting dynamically based on the social relationship.
  arXiv  Detail & Related papers  (2024-10-10T12:30:56Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization [63.554226552130054]
  Generalization poses a significant challenge in Multi-agent Reinforcement Learning (MARL) The extent to which an agent is influenced by unseen co-players depends on the agent's policy and the specific scenario. We present the Level of Influence (LoI), a metric quantifying the interaction intensity among agents within a given scenario and environment.
  arXiv  Detail & Related papers  (2023-10-11T06:09:26Z)
• Mimicking Better by Matching the Approximate Action Distribution [48.95048003354255]
  We introduce MAAD, a novel, sample-efficient on-policy algorithm for Imitation Learning from Observations. We show that it requires considerable fewer interactions to achieve expert performance, outperforming current state-of-the-art on-policy methods.
  arXiv  Detail & Related papers  (2023-06-16T12:43:47Z)
• Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teaming [0.0]
  We show that reformulating an agent's policy to be conditional on the policies of its teammates inherently maximizes Mutual Information (MI) lower-bound when optimizing under Policy Gradient (PG) Our approach, InfoPG, outperforms baselines in learning emergent collaborative behaviors and sets the state-of-the-art in decentralized cooperative MARL tasks.
  arXiv  Detail & Related papers  (2022-01-20T22:54:32Z)
• A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation [61.740187363451746]
  Marginalized importance sampling (MIS) measures the density ratio between the state-action occupancy of a target policy and that of a sampling distribution. We bridge the gap between MIS and deep reinforcement learning by observing that the density ratio can be computed from the successor representation of the target policy. We evaluate the empirical performance of our approach on a variety of challenging Atari and MuJoCo environments.
  arXiv  Detail & Related papers  (2021-06-12T20:21:38Z)
• Balancing Rational and Other-Regarding Preferences in Cooperative-Competitive Environments [4.705291741591329]
  Mixed environments are notorious for the conflicts of selfish and social interests. We propose BAROCCO to balance individual and social incentives. Our meta-algorithm is compatible with both Q-learning and Actor-Critic frameworks.
  arXiv  Detail & Related papers  (2021-02-24T14:35:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.