MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate

• URL: http://arxiv.org/abs/2609.04841v1
• Date: Fri, 04 Sep 2026 07:58:33 GMT
• Title: MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate
• Abstract summary: We introduce MABPD (Multi-Agent Bias Probing & Detection), a principled, training-free alternative to supervised classification.<n>SAD implements a domain-motivated asymmetric burden of proof---biased claims without grounded textual evidence carry zero weight.<n>Ablation confirms that this structured deliberation, not mere agent parallelism, drives performance.
• Score: 0.06999740786886537
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Media bias in news articles operates through subtle linguistic cues---loaded language, selective framing, and strategic omission---that resist single-model detection and have traditionally required large annotated corpora for supervised training. We ask whether structured multi-agent deliberation can serve as a principled, training-free alternative to supervised classification for this task. We introduce MABPD (Multi-Agent Bias Probing & Detection), a pipeline in which three specialized LLM agents analyze an article from complementary perspectives and resolve disagreements through a Structured Argument Debate (SAD) protocol. SAD implements a domain-motivated asymmetric burden of proof---biased claims without grounded textual evidence carry zero weight---combined with role-weighted voting and post-consensus verification, replacing task-specific supervised decision boundaries with explicit deliberative structure. Ablation confirms that this structured deliberation, not mere agent parallelism, drives performance: removing the debate module reduces F1 by up to 10.6 points. On the BABE benchmark (4,121 expert-annotated sentences), MABPD achieves 83.4% macro F1 on the held-out test split---within 0.7 percentage points (pp) of the supervised SOTA (MAGPIE, 84.1% macro F1; Horych et al., 2024)---without any task-specific training or threshold tuning on annotated data. Cross-dataset evaluation on the SemEval 2019 HyperPartisan corpus (644 articles) yields 75.0% zero-shot accuracy, within 7.2 pp of the supervised SOTA accuracy (82.2%; Kiesel et al. 2019), confirming transfer across annotation regimes. We release the full pipeline and evaluation code.

Related papers

• The Deception Delta: Adversarial Evaluation of LLM-Based Smart Contract Bytecode Forensics [0.0]
  Large language models are increasingly used in blockchain forensic investigations.<n> adversarial deception reduces drain detection by 20.0 percentage points.<n>Models correctly describe the hidden drain mechanism but accept the contract's framing and dismiss it as benign.
  arXiv  Detail & Related papers  (2026-09-12T18:45:24Z)
• Grounded Checklist Partial Credit for Agent Skill Trajectories [5.028537745773042]
  Grounded Checklist Partial Credit (GCPC) is a human-governed partial-credit evaluation of agent trajectories.<n>Human evaluation on 96 trajectories from 12 tasks shows that G CPC aligns more closely with human assessments of progress.<n>The pipeline also transfers to Terminal-Bench and SWE-bench, demonstrating applicability beyond skill-conditioned evaluation.
  arXiv  Detail & Related papers  (2026-08-26T06:17:19Z)
• Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agents [5.689042186242701]
  Decision rules that enterprise experts apply tacitly can be systematically recovered and improved through iterative error analysis.<n>We present textbfTrace2Policy, whose core mechanism is textbfEISR.<n>Each round executes the rules on a validation set, clusters errors by root cause into MISSING, WRONG, or CONFLICT types, applies targeted patches, and commits only those that pass a regression gate.
  arXiv  Detail & Related papers  (2026-06-09T06:05:29Z)
• Automated Benchmark Auditing for AI Agents and Large Language Models [46.03841647776303]
  Auto Benchmark Audit (ABA) is an agentic framework that systematically audits individual benchmark tasks.<n>We run ABA on a collection of frontier LLM benchmarks and previous NeurIPS publications, totaling 168 benchmarks across nine domains.<n>ABA identifies critical issues including ambiguous task design, execution environment conflicts, and incorrect ground truths in over 25.7% of the evaluated tasks.
  arXiv  Detail & Related papers  (2026-05-25T17:44:21Z)
• Structured Uncertainty guided Clarification for LLM Agents [126.26213027785813]
  LLM agents extend large language models with tool-calling capabilities, but ambiguous user instructions often lead to incorrect invocations and task failures.<n>We introduce a principled formulation of structured uncertainty over tool-call parameters, modeling joint tool-argument clarification as a POMDP with Expected Value of Perfect Information (EVPI) objective for optimal question selection and aspect-based cost modeling to prevent redundancy.<n>Our SAGE-Agent leverages this structured uncertainty to achieve superior efficiency: increasing coverage on ambiguous tasks by 7-39% while reducing clarification questions by 1.5-2.7$times$ compared to strong prompting and uncertainty-based baselines.
  arXiv  Detail & Related papers  (2025-11-11T21:50:44Z)
• FOR-Prompting: From Objection to Revision via an Asymmetric Prompting Protocol [7.765950922513099]
  Reasoning protocols organize internal deliberation but lack an explicit mechanism for external questioning that elicits self-revision.<n>We present FOR-Prompting, an asymmetric protocol where a Defender proposes an answer, an Objectioner raises question-style objections with no direct fixes, and a Host enforces consistency and closure.<n>On GSM8K we observe about a 22% point gain over single-prompt and accuracy on par with CoT, with more than 10% higher ratings in reasoning and coherence from a uniform GPT 4.1 judge.
  arXiv  Detail & Related papers  (2025-10-02T04:57:58Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Internalizing Self-Consistency in Language Models: Multi-Agent Consensus Alignment [22.305033366660187]
  Language Models (LMs) are inconsistent reasoners, often generating contradictory responses to identical prompts.<n>We formalize self-consistency as an intrinsic property of well-aligned reasoning models and introduce Multi-Agent Consensus Alignment (MACA)<n>MACA enables agents to teach themselves to be more decisive and concise, and better leverage peer insights in multi-agent settings without external supervision.
  arXiv  Detail & Related papers  (2025-09-18T17:27:28Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)
• The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Models [69.798277882245]
  We introduce Unsupervised Prefix Fine-Tuning (UPFT) to enhance large language models' reasoning efficiency.<n>UPFT removes the need for labeled data or exhaustive sampling.<n> Experiments show that UPFT matches the performance of supervised methods.
  arXiv  Detail & Related papers  (2025-03-04T18:56:03Z)
• ADEPT: A DEbiasing PrompT Framework [64.54665501064659]
  Finetuning is an applicable approach for debiasing contextualized word embeddings.<n> discrete prompts with semantic meanings have shown to be effective in debiasing tasks.<n>We propose ADEPT, a method to debias PLMs using prompt tuning while maintaining the delicate balance between removing biases and ensuring representation ability.
  arXiv  Detail & Related papers  (2022-11-10T08:41:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.