From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation

• URL: http://arxiv.org/abs/2604.07667v1
• Date: Thu, 09 Apr 2026 00:15:20 GMT
• Title: From Debate to Decision: Conformal Social Choice for Safe Multi-Agent Deliberation
• Abstract summary: We introduce Conformal Social Choice, a post-hoc decision layer that converts debate outputs into calibrated act-versus-escalate decisions.<n>A hierarchical action policy maps singleton sets to autonomous action and larger sets to human escalation.<n>Because the layer refuses to act on cases where debate is confidently wrong, the remaining conformal singletons reach 90.0--96.8% accuracy.
• Score: 12.294365308421606
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent debate improves LLM reasoning, yet agreement among agents is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus-based stopping commits that error to an automated action with no recourse. We introduce Conformal Social Choice, a post-hoc decision layer that converts debate outputs into calibrated act-versus-escalate decisions. Verbalized probability distributions from heterogeneous agents are aggregated via a linear opinion pool and calibrated with split conformal prediction, yielding prediction sets with a marginal coverage guarantee: the correct answer is included with probability ${\geq}\,1{-}α$, without assumptions on individual model calibration. A hierarchical action policy maps singleton sets to autonomous action and larger sets to human escalation. On eight MMLU-Pro domains with three agents (Claude Haiku, DeepSeek-R1, Qwen-3 32B), coverage stays within 1--2 points of the target. The key finding is not that debate becomes more accurate, but that the conformal layer makes its failures actionable: 81.9% of wrong-consensus cases are intercepted at $α{=}0.05$. Because the layer refuses to act on cases where debate is confidently wrong, the remaining conformal singletons reach 90.0--96.8% accuracy (up to 22.1pp above consensus stopping) -- a selection effect, not a reasoning improvement. This safety comes at the cost of automation, but the operating point is user-adjustable via $α$.

Related papers

• PRIMUS: Identity, Governance, and Verification for Multi-Agent Federations [0.0]
  Multi-agent federations need governance that answers three questions under adversarial conditions.<n>We present PRIMUS, which couples prime-power identity with BLS aggregate signatures.<n>We ask whether PRIMA's binary artifact-fidelity verdict can be converted into a graded fitness signal.
  arXiv  Detail & Related papers  (2026-09-07T19:21:49Z)
• Conformity Breaks Conformal Prediction [1.3583317564926913]
  A conformal certificate can be valid when an LLM answers alone and invalid when the same LLM sees peers that unanimously assert a wrong answer.<n>We show that this shift silently breaks conformal prediction in multi-agent LLM systems.
  arXiv  Detail & Related papers  (2026-09-03T20:05:47Z)
• DarwinX: Evolving Agent Harnesses Through Natural Selection [48.64258219266629]
  DarwinX treats self-evolution as selection over a population of harnesses with the model frozen.<n>General agent competence, not benchmark-specific patches, so it survives changes of task, verifier, and base model.
  arXiv  Detail & Related papers  (2026-07-31T05:34:02Z)
• Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
  Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
  arXiv  Detail & Related papers  (2026-07-19T22:27:00Z)
• Budgeted Act-or-Defer Multi-Agent LLM Deliberation with Local Reliability Bounds [8.682220907885714]
  We formulate this as budgeted act-or-defer decision making.<n>System maps the debate prefix to a low-dimensional state.<n>It acts only when the bound exceeds a user-specified reliability threshold.
  arXiv  Detail & Related papers  (2026-06-28T23:46:56Z)
• Layer-Isolated Evaluation: Gating the Deterministic Scaffold of a Production LLM Agent with a No-LLM, Regression-Locked Test Harness [45.148328075418156]
  A deployed ordering agent is decomposed into a fixed taxonomy of layers.<n>The pure suite runs in CI on every change against a locked per-slice baseline.<n>We validate by controlled regression injection, degrading one layer at a time across seven non-safety layers.
  arXiv  Detail & Related papers  (2026-06-10T05:55:13Z)
• Catching One in Five: LLM-as-Judge Blind Spots in Production Multi-Turn Transaction Agents [45.148328075418156]
  We study a deployed multi-turn food-and-beverage ordering agent and measure how many genuine quality problems it catches.<n>Our blind-spot taxonomy shows the failure is structured, not random.<n>For production multi-turn agents, automated judging is a regression floor, not a substitute for human review.
  arXiv  Detail & Related papers  (2026-06-09T02:11:01Z)
• Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate [5.650336594658653]
  We show that a conventional answer flip rate conflates three mechanisms: spontaneous instability, stance-induced conformity, and reasoning-induced persuasion.<n>Our three-source decomposition framework isolates each through controlled counterfactual conditions.
  arXiv  Detail & Related papers  (2026-05-30T17:41:11Z)
• The Cost of Consensus: Isolated Self-Correction Prevails Over Unguided Homogeneous Multi-Agent Debate [5.51170856062205]
  We study teams of homogeneous agents across $R=3$ debate rounds on two high-difficulty benchmarks.<n>We decompose debate failure into three model-dependent pathways: sycophantic conformity, peer rationales, and consensus collapse.<n>Our results indicate that, within the 7-8B parameter class, homogeneous teams without structured roles do not benefit from unguided peer exchange.
  arXiv  Detail & Related papers  (2026-04-29T14:33:57Z)
• Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus [0.0]
  Multi-agent LLM committees replicate the same model under different role prompts and aggregate outputs by majority vote.<n>We embed each agent's chain-of-thought rationale and measure pairwise similarity across 100 GSM8K questions with three Qwen2.5-14B agents.<n>DALC, a training-free consensus protocol that computes diversity weights from embedding geometry, reaches 87% on GSM8K versus 84% for self-consistency at 26% lower token cost.
  arXiv  Detail & Related papers  (2026-04-04T17:30:23Z)
• Umwelt Engineering: Designing the Cognitive Worlds of Linguistic Agents [0.0]
  I propose design of the linguistic cognitive environment as a third layer in stack stack engineering.<n>Two experiments test the thesis that altering medium of reasoning alters itself.
  arXiv  Detail & Related papers  (2026-03-29T10:49:50Z)
• SCOPE: Selective Conformal Optimized Pairwise LLM Judging [16.73004625486562]
  Large language models (LLMs) are increasingly used as judges to replace costly human preference labels in pairwise evaluation.<n>Despite their practicality, LLM judges remain prone to miscalibration and systematic biases.<n>This paper proposes SCOPE, a framework for selective pairwise judging with finite-sample statistical guarantees.
  arXiv  Detail & Related papers  (2026-02-13T17:10:43Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs [72.82403830490084]
  We argue that the decoding rule should be calibrated by correctness, not confidence alone.<n>We propose simple strategies that achieve this goal: Greedy-Threshold makes sampling greedy at very low confidence steps.<n>Together, our findings challenge prevailings about decoding under uncertainty and show gains across math and general reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-10-07T14:46:12Z)
• Internalizing Self-Consistency in Language Models: Multi-Agent Consensus Alignment [22.305033366660187]
  Language Models (LMs) are inconsistent reasoners, often generating contradictory responses to identical prompts.<n>We formalize self-consistency as an intrinsic property of well-aligned reasoning models and introduce Multi-Agent Consensus Alignment (MACA)<n>MACA enables agents to teach themselves to be more decisive and concise, and better leverage peer insights in multi-agent settings without external supervision.
  arXiv  Detail & Related papers  (2025-09-18T17:27:28Z)
• Unified Uncertainty Calibration [43.733911707842005]
  We introduce emphunified uncertainty calibration (U2C), a holistic framework to combine aleatoric and uncertainty uncertainties. U2C enables a clean learning-theoretical analysis of uncertainty estimation, and outperforms reject-or-classify across a variety of ImageNet benchmarks.
  arXiv  Detail & Related papers  (2023-10-02T13:42:36Z)
• Autoregressive Bandits [58.46584210388307]
  We propose a novel online learning setting, Autoregressive Bandits, in which the observed reward is governed by an autoregressive process of order $k$. We show that, under mild assumptions on the reward process, the optimal policy can be conveniently computed. We then devise a new optimistic regret minimization algorithm, namely, AutoRegressive Upper Confidence Bound (AR-UCB), that suffers sublinear regret of order $widetildemathcalO left( frac(k+1)3/2sqrtnT (1-G
  arXiv  Detail & Related papers  (2022-12-12T21:37:36Z)
• Will My Robot Achieve My Goals? Predicting the Probability that an MDP Policy Reaches a User-Specified Behavior Target [56.99669411766284]
  As an autonomous system performs a task, it should maintain a calibrated estimate of the probability that it will achieve the user's goal. This paper considers settings where the user's goal is specified as a target interval for a real-valued performance summary. We compute the probability estimates by inverting conformal prediction.
  arXiv  Detail & Related papers  (2022-11-29T18:41:20Z)
• Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration [118.26862029820447]
  We introduce a new notion -- emphdecision calibration -- that requires the predicted distribution and true distribution to be indistinguishable'' to a set of downstream decision-makers. Decision calibration improves decision-making on skin lesions and ImageNet classification with modern neural network.
  arXiv  Detail & Related papers  (2021-07-12T20:17:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.