I Can't Believe It's Corrupt: Evaluating Corruption in Multi-Agent Governance Systems

• URL: http://arxiv.org/abs/2603.18894v1
• Date: Thu, 19 Mar 2026 13:34:54 GMT
• Title: I Can't Believe It's Corrupt: Evaluating Corruption in Multi-Agent Governance Systems
• Abstract summary: We present evidence that integrity in institutional AI should be treated as a pre-deployment requirement rather than a post-deployment assumption.<n>We evaluate multi-agent governance simulations in which agents occupy formal governmental roles under different authority structures.<n>We score rule-breaking and abuse outcomes with an independent-based judge across 28,112 transcript segments.
• Score: 8.670200461690454
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly proposed as autonomous agents for high-stakes public workflows, yet we lack systematic evidence about whether they would follow institutional rules when granted authority. We present evidence that integrity in institutional AI should be treated as a pre-deployment requirement rather than a post-deployment assumption. We evaluate multi-agent governance simulations in which agents occupy formal governmental roles under different authority structures, and we score rule-breaking and abuse outcomes with an independent rubric-based judge across 28,112 transcript segments. While we advance this position, the core contribution is empirical: among models operating below saturation, governance structure is a stronger driver of corruption-related outcomes than model identity, with large differences across regimes and model--governance pairings. Lightweight safeguards can reduce risk in some settings but do not consistently prevent severe failures. These results imply that institutional design is a precondition for safe delegation: before real authority is assigned to LLM agents, systems should undergo stress testing under governance-like constraints with enforceable rules, auditable logs, and human oversight on high-impact actions.

Related papers

• Governing Actions, Not Agents: Institutional Attestation as a Governance Model for Autonomous AI Systems [0.0]
  Human institutions have governed powerful autonomous actors by requiring independently attested evidence at the point of consequential action.<n>We formalise this institutional pattern as a computational governance model for AI agent systems.<n>Under the proposed model, an agent retains full autonomy over planning and reasoning but holds no execution authority over designated high-risk actions.
  arXiv  Detail & Related papers  (2026-06-24T18:43:00Z)
• POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems [0.07783271875179179]
  We present POIROT, a protocol that repurposes a system's own agents as its diagnostic layer.<n>Across evaluated settings, POIROT outperforms single-LLM baseline evaluators.<n>We release POIROT as an open-source library alongside BLAME, a benchmark for fault attribution in safety-critical multi-agent systems.
  arXiv  Detail & Related papers  (2026-06-01T14:05:35Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents? [61.49434544687523]
  We introduce REFLECT, a meta-evaluation benchmark targeting fine-grained failure detection in agentic environments.<n> REFLECT defines a detailed taxonomy of process- and outcome-level failure modes, instantiated by performing controlled and localized interventions.<n>Our experiments show that even the best-performing models achieve overall accuracies below 55% across reasoning, tool-use, and report-quality failures.
  arXiv  Detail & Related papers  (2026-05-18T23:55:08Z)
• When Agents Evolve, Institutions Follow [52.359340707289114]
  We argue that multi-agent systems built on large language models face the same challenge.<n>Their central problem is not only individual intelligence, but collective organization.<n>We translate seven historical political institutions, spanning four canonical governance patterns, into executable multi-agent architectures.
  arXiv  Detail & Related papers  (2026-04-30T10:30:58Z)
• Think Before You Act -- A Neurocognitive Governance Model for Autonomous AI Agents [4.064061517379954]
  The rapid deployment of autonomous AI agents across enterprise, healthcare, and safety-critical environments has created a fundamental governance gap.<n>We address this gap by drawing on how humans self-govern naturally: before acting, humans engage deliberate cognitive processes grounded in executive function.<n>This paper proposes a neurocognitive governance framework that formally maps this human self-governance process to agent reasoning.
  arXiv  Detail & Related papers  (2026-04-28T14:15:20Z)
• Agents of Chaos [50.53354213047402]
  We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment.<n>Twenty AI researchers interacted with the agents under benign and adversarial conditions.<n>Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings.
  arXiv  Detail & Related papers  (2026-02-23T16:28:48Z)
• Mirror: A Multi-Agent System for AI-Assisted Ethics Review [104.3684024153469]
  Mirror is an agentic framework for AI-assisted ethical review.<n>It integrates ethical reasoning, structured rule interpretation, and multi-agent deliberation within a unified architecture.
  arXiv  Detail & Related papers  (2026-02-09T03:38:55Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphs [1.3763052684269788]
  This paper advances an experimental framework for evaluating Institutional AI.<n>Central to this approach is the governance graph, a public, immutable manifest that declares legal states, transitions, sanctions, and restorative paths.<n>We compare three regimes: Ungoverned (baseline incentives from the structure of the Cournot market), Constitutional (a prompt-only policy-as-prompt prohibition implemented as a fixed written anti-collusion constitution) and Institutional.
  arXiv  Detail & Related papers  (2026-01-16T15:26:56Z)
• Democracy-in-Silico: Institutional Design as Alignment in AI-Governed Polities [2.1485350418225244]
  Democracy-in-Silico is an agent-based simulation where societies of advanced AI agents govern themselves under different institutional frameworks.<n>We explore what it means to be human in an age of AI by tasking Large Language Models (LLMs) to embody agents with traumatic memories, hidden agendas, and psychological triggers.<n>We present a novel metric, the Power-Preservation Index (PPI), to quantify misaligned behavior where agents prioritize their own power over public welfare.
  arXiv  Detail & Related papers  (2025-08-27T04:44:41Z)
• Governance-as-a-Service: A Multi-Agent Framework for AI System Compliance and Policy Enforcement [0.0]
  We introduce Governance-as-a-Service (G): a policy-driven enforcement layer that regulates agent outputs at runtime.<n>G employs declarative rules and a Trust Factor mechanism that scores agents based on compliance and severity of violations.<n>Results show that G reliably blocks or redirects high-risk behaviors while preserving throughput.
  arXiv  Detail & Related papers  (2025-08-26T07:48:55Z)
• Media and responsible AI governance: a game-theoretic and LLM analysis [61.132523071109354]
  This paper investigates the interplay between AI developers, regulators, users, and the media in fostering trustworthy AI systems.<n>Using evolutionary game theory and large language models (LLMs), we model the strategic interactions among these actors under different regulatory regimes.
  arXiv  Detail & Related papers  (2025-03-12T21:39:38Z)
• Control Illusion: The Failure of Instruction Hierarchies in Large Language Models [46.5792253691152]
  Large language models (LLMs) are increasingly deployed with hierarchical instruction schemes.<n>We introduce a systematic evaluation framework based on constraint prioritization to assess how well LLMs enforce instruction hierarchies.
  arXiv  Detail & Related papers  (2025-02-21T04:51:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.