CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas

• URL: http://arxiv.org/abs/2604.15267v1
• Date: Thu, 16 Apr 2026 17:40:30 GMT
• Title: CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas
• Abstract summary: LLMs with stronger reasoning capabilities behave _less_ cooperatively in mixed-motive games.<n>Our experiments show that recent models consistently defect in single-shot social dilemmas.<n>Contracting and mediation are most effective in achieving cooperative outcomes between capable LLM models.
• Score: 37.9285901060329
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is increasingly important that LLM agents interact effectively and safely with other goal-pursuing agents, yet, recent works report the opposite trend: LLMs with stronger reasoning capabilities behave _less_ cooperatively in mixed-motive games such as the prisoner's dilemma and public goods settings. Indeed, our experiments show that recent models -- with or without reasoning enabled -- consistently defect in single-shot social dilemmas. To tackle this safety concern, we present the first comparative study of game-theoretic mechanisms that are designed to enable cooperative outcomes between rational agents _in equilibrium_. Across four social dilemmas testing distinct components of robust cooperation, we evaluate the following mechanisms: (1) repeating the game for many rounds, (2) reputation systems, (3) third-party mediators to delegate decision making to, and (4) contract agreements for outcome-conditional payments between players. Among our findings, we establish that contracting and mediation are most effective in achieving cooperative outcomes between capable LLM models, and that repetition-induced cooperation deteriorates drastically when co-players vary. Moreover, we demonstrate that these cooperation mechanisms become _more effective_ under evolutionary pressures to maximize individual payoffs.

Related papers

• Fairness over Equality: Correcting Social Incentives in Asymmetric Sequential Social Dilemmas [14.346045345458416]
  Sequential Social Dilemmas (SSDs) provide a key framework for studying how cooperation emerges when individual incentives conflict with collective welfare.<n>Most existing methods assume that agents face identical incentives in the dilemma and require continuous access to global information about other agents to assess fairness.<n>We introduce asymmetric variants of well-known SSD environments and examine how natural differences between agents influence cooperation dynamics.
  arXiv  Detail & Related papers  (2026-02-17T07:31:20Z)
• Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia [100.74015791021044]
  Large Language Model (LLM) agents have demonstrated impressive capabilities for social interaction.<n>Existing evaluation methods fail to measure how well these capabilities generalize to novel social situations.<n>We present empirical results from the NeurIPS 2024 Concordia Contest, where agents were evaluated on their ability to achieve mutual gains.
  arXiv  Detail & Related papers  (2025-12-03T00:11:05Z)
• The Collaboration Gap [28.553543260404425]
  We propose a collaborative maze-solving benchmark that (i) isolates collaborative capabilities, (ii) modulates problem complexity, (iii) enables scalable automated grading, and (iv) imposes no output constraintsformat.<n>Using this framework, we evaluate 32 leading open- and closed-source models in solo, homogeneous, and heterogeneous pairings.<n>Our results reveal a "collaboration gap": models that perform well solo often degrade substantially when required to collaborate.
  arXiv  Detail & Related papers  (2025-11-04T16:10:57Z)
• How large language models judge and influence human cooperation [82.07571393247476]
  We assess how state-of-the-art language models judge cooperative actions.<n>We observe a remarkable agreement in evaluating cooperation against good opponents.<n>We show that the differences revealed between models can significantly impact the prevalence of cooperation.
  arXiv  Detail & Related papers  (2025-06-30T09:14:42Z)
• Corrupted by Reasoning: Reasoning Language Models Become Free-Riders in Public Goods Games [87.5673042805229]
  How large language models balance self-interest and collective well-being is a critical challenge for ensuring alignment, robustness, and safe deployment.<n>We adapt a public goods game with institutional choice from behavioral economics, allowing us to observe how different LLMs navigate social dilemmas.<n>Surprisingly, we find that reasoning LLMs, such as the o1 series, struggle significantly with cooperation.
  arXiv  Detail & Related papers  (2025-06-29T15:02:47Z)
• Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents [101.17919953243107]
  GovSim is a generative simulation platform designed to study strategic interactions and cooperative decision-making in large language models (LLMs)<n>We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in GovSim, with the highest survival rate below 54%.<n>We show that agents that leverage "Universalization"-based reasoning, a theory of moral thinking, are able to achieve significantly better sustainability.
  arXiv  Detail & Related papers  (2024-04-25T15:59:16Z)
• Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View [60.80731090755224]
  This paper probes the collaboration mechanisms among contemporary NLP systems by practical experiments with theoretical insights. We fabricate four unique societies' comprised of LLM agents, where each agent is characterized by a specific trait' (easy-going or overconfident) and engages in collaboration with a distinct thinking pattern' (debate or reflection) Our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring social psychology theories.
  arXiv  Detail & Related papers  (2023-10-03T15:05:52Z)
• Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL [20.22747008079794]
  We study the dynamics of a second-order social dilemma resulting from incentivizing mechanisms. We find that a typical tendency of humans, called homophily, can solve the problem. We propose a novel learning framework to encourage incentive homophily.
  arXiv  Detail & Related papers  (2021-04-23T08:00:45Z)
• Balancing Rational and Other-Regarding Preferences in Cooperative-Competitive Environments [4.705291741591329]
  Mixed environments are notorious for the conflicts of selfish and social interests. We propose BAROCCO to balance individual and social incentives. Our meta-algorithm is compatible with both Q-learning and Actor-Critic frameworks.
  arXiv  Detail & Related papers  (2021-02-24T14:35:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.