Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs

• URL: http://arxiv.org/abs/2601.11369v2
• Date: Tue, 20 Jan 2026 12:10:21 GMT
• Title: Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs
• Authors: Marcantonio Bracale Syrnikov, Federico Pierucci, Marcello Galisai, Matteo Prandi, Piercosma Bisconti, Francesco Giarrusso, Olga Sorokoletova, Vincenzo Suriani, Daniele Nardi,
• Abstract summary: This paper advances an experimental framework for evaluating Institutional AI.<n>Central to this approach is the governance graph, a public, immutable manifest that declares legal states, transitions, sanctions, and restorative paths.<n>We compare three regimes: Ungoverned (baseline incentives from the structure of the Cournot market), Constitutional (a prompt-only policy-as-prompt prohibition implemented as a fixed written anti-collusion constitution) and Institutional.
• Score: 1.3763052684269788
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent LLM ensembles can converge on coordinated, socially harmful equilibria. This paper advances an experimental framework for evaluating Institutional AI, our system-level approach to AI alignment that reframes alignment from preference engineering in agent-space to mechanism design in institution-space. Central to this approach is the governance graph, a public, immutable manifest that declares legal states, transitions, sanctions, and restorative paths; an Oracle/Controller runtime interprets this manifest, attaching enforceable consequences to evidence of coordination while recording a cryptographically keyed, append-only governance log for audit and provenance. We apply the Institutional AI framework to govern the Cournot collusion case documented by prior work and compare three regimes: Ungoverned (baseline incentives from the structure of the Cournot market), Constitutional (a prompt-only policy-as-prompt prohibition implemented as a fixed written anti-collusion constitution, and Institutional (governance-graph-based). Across six model configurations including cross-provider pairs (N=90 runs/condition), the Institutional regime produces large reductions in collusion: mean tier falls from 3.1 to 1.8 (Cohen's d=1.28), and severe-collusion incidence drops from 50% to 5.6%. The prompt-only Constitutional baseline yields no reliable improvement, illustrating that declarative prohibitions do not bind under optimisation pressure. These results suggest that multi-agent alignment may benefit from being framed as an institutional design problem, where governance graphs can provide a tractable abstraction for alignment-relevant collective behavior.

Related papers

• Incentive Aware AI Regulations: A Credal Characterisation [14.228416693145649]
  High-stakes ML applications demand strict regulations, but strategic ML providers often evade them to lower development costs.<n>We introduce regulation mechanisms: a framework that maps empirical evidence from models to a license for some market share.<n>We prove that a mechanism has perfect market outcome if and only if the set of non-compliant distributions forms a credal set of probability measures.
  arXiv  Detail & Related papers  (2026-03-05T13:42:19Z)
• Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science [70.3658845234978]
  Large Language Models (LLMs) have greatly extended the capabilities of Multi-Agent Systems (MAS)<n>Despite this rapid progress, the field still relies heavily on empirical trial-and-error.<n>This bottleneck stems from the ambiguity of attribution.<n>We propose a factor attribution paradigm to systematically identify collaboration-driving factors.
  arXiv  Detail & Related papers  (2026-02-05T04:19:52Z)
• Self-Evolving Coordination Protocol in Multi-Agent AI Systems: An Exploratory Systems Feasibility Study [0.0]
  Self-Evolving Coordination Protocols (SECP)<n>SECP: coordination protocols that permit limited, externally validated self-modification.<n>This paper presents an exploratory systems feasibility study of Self-Evolving Coordination Protocols.
  arXiv  Detail & Related papers  (2026-02-02T14:45:04Z)
• Autonomous Chain-of-Thought Distillation for Graph-Based Fraud Detection [73.9189065770752]
  Graph-based fraud detection on text-attributed graphs (TAGs) requires jointly modeling rich textual semantics and relational dependencies.<n>We propose FraudCoT, a unified framework that advances TAG-based fraud detection through autonomous, graph-aware chain-of-thought (CoT) reasoning and scalable LLM-GNN co-training.
  arXiv  Detail & Related papers  (2026-01-30T13:12:12Z)
• Preventing the Collapse of Peer Review Requires Verification-First AI [49.995126139461085]
  We propose truth-coupling, i.e. how tightly venue scores track latent scientific truth.<n>We formalize two forces that drive a phase transition toward proxy-sovereign evaluation.
  arXiv  Detail & Related papers  (2026-01-23T17:17:32Z)
• Institutional AI: A Governance Framework for Distributional AGI Safety [1.3763052684269788]
  We identify three structural problems that emerge from core properties of AI models.<n>The solution is Institutional AI, a system-level approach that treats alignment as a question of effective governance of AI agent collectives.
  arXiv  Detail & Related papers  (2026-01-15T17:08:26Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions [51.56484100374058]
  Current large language models (LLMs) excel in verifiable domains where outputs can be checked before action but prove less reliable for high-stakes strategic decisions with uncertain outcomes.<n>This gap, driven by mutually cognitive biases in both humans and artificial intelligence (AI) systems, threatens the defensibility of valuations and sustainability of investments in the sector.<n>This report describes a framework emerging from systematic qualitative assessment across 7 frontier-grade LLMs and 3 market-facing venture vignettes under time pressure.
  arXiv  Detail & Related papers  (2025-11-10T22:24:21Z)
• Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities [2.1485350418225244]
  Democracy-in-Silico is an agent-based simulation where societies of advanced AI agents govern themselves under different institutional frameworks.<n>We explore what it means to be human in an age of AI by tasking Large Language Models (LLMs) to embody agents with traumatic memories, hidden agendas, and psychological triggers.<n>We present a novel metric, the Power-Preservation Index (PPI), to quantify misaligned behavior where agents prioritize their own power over public welfare.
  arXiv  Detail & Related papers  (2025-08-27T04:44:41Z)
• Governance-as-a-Service: A Multi-Agent Framework for AI System Compliance and Policy Enforcement [0.0]
  We introduce Governance-as-a-Service (G): a policy-driven enforcement layer that regulates agent outputs at runtime.<n>G employs declarative rules and a Trust Factor mechanism that scores agents based on compliance and severity of violations.<n>Results show that G reliably blocks or redirects high-risk behaviors while preserving throughput.
  arXiv  Detail & Related papers  (2025-08-26T07:48:55Z)
• Watermarking Without Standards Is Not AI Governance [46.71493672772134]
  We argue that current implementations risk serving as symbolic compliance rather than delivering effective oversight.<n>We propose a three-layer framework encompassing technical standards, audit infrastructure, and enforcement mechanisms.
  arXiv  Detail & Related papers  (2025-05-27T18:10:04Z)
• Media and responsible AI governance: a game-theoretic and LLM analysis [61.132523071109354]
  This paper investigates the interplay between AI developers, regulators, users, and the media in fostering trustworthy AI systems.<n>Using evolutionary game theory and large language models (LLMs), we model the strategic interactions among these actors under different regulatory regimes.
  arXiv  Detail & Related papers  (2025-03-12T21:39:38Z)
• AGI, Governments, and Free Societies [0.0]
  We argue that AGI poses distinct risks of pushing societies toward either a 'despotic Leviathan' or an 'absent Leviathan'<n>We analyze how these dynamics could unfold through three key channels.<n> Enhanced state capacity through AGI could enable unprecedented surveillance and control, potentially entrenching authoritarian practices.<n>Conversely, rapid diffusion of AGI capabilities to non-state actors could undermine state legitimacy and governability.
  arXiv  Detail & Related papers  (2025-02-14T03:55:38Z)
• Unsupervised Full Constituency Parsing with Neighboring Distribution Divergence [48.69930912510414]
  We propose an unsupervised and training-free labeling procedure by exploiting the property of a recently introduced metric. For implementation, we develop NDD into Dual POS-NDD and build "molds" to detect constituents and their labels in sentences. We show that DP-NDD not only labels constituents precisely but also inducts more accurate unlabeled constituency trees than all previous unsupervised methods with simpler rules.
  arXiv  Detail & Related papers  (2021-10-29T17:27:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.