MAC: Multi-Agent Constitution Learning

• URL: http://arxiv.org/abs/2603.15968v1
• Date: Mon, 16 Mar 2026 22:42:07 GMT
• Title: MAC: Multi-Agent Constitution Learning
• Abstract summary: Constitutional AI is a method to oversee and control LLMs based on a set of rules written in natural language.<n>Existing LLM-based prompt methods are ineffective at learning constitutions since they require many labeled examples and lack structure in the optimized prompts.<n>We propose Multi-Agent Constitutional Learning (MAC), which optimize over structured prompts represented as sets of rules.
• Score: 13.313180901776816
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Constitutional AI is a method to oversee and control LLMs based on a set of rules written in natural language. These rules are typically written by human experts, but could in principle be learned automatically given sufficient training data for the desired behavior. Existing LLM-based prompt optimizers attempt this but are ineffective at learning constitutions since (i) they require many labeled examples and (ii) lack structure in the optimized prompts, leading to diminishing improvements as prompt size grows. To address these limitations, we propose Multi-Agent Constitutional Learning (MAC), which optimizes over structured prompts represented as sets of rules using a network of agents with specialized tasks to accept, edit, or reject rule updates. We also present MAC+, which improves performance by training agents on successful trajectories to reinforce updates leading to higher reward. We evaluate MAC on tagging Personally Identifiable Information (PII), a classification task with limited labels where interpretability is critical, and demonstrate that it generalizes to other agentic tasks such as tool calling. MAC outperforms recent prompt optimization methods by over 50%, produces human-readable and auditable rule sets, and achieves performance comparable to supervised fine-tuning and GRPO without requiring parameter updates.

Related papers

• TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter [1.6003479913250962]
  Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential.<n>This work addresses the challenge by introducing a Task-Aware Prompt Rewriter (TAPR), a model that reformulates user prompts into task-optimized prompts.
  arXiv  Detail & Related papers  (2026-07-17T15:37:07Z)
• RIMRULE: Improving Tool-Using Language Agents via MDL-Guided Rule Learning [31.8598046524742]
  RIMRULE is a neuro-symbolic approach for large language models (LLMs) adaptation based on dynamic rule injection.<n> Compact, interpretable rules are distilled from failure traces and injected into the prompt during inference to improve task performance.
  arXiv  Detail & Related papers  (2025-12-31T19:40:10Z)
• Reasoning Up the Instruction Ladder for Controllable Language Models [26.068755167791505]
  Large language model (LLM) based systems take on high-stakes roles in real-world decision-making.<n>Enforcement of an instruction hierarchy (IH) in LLMs is critical for the reliability and controllability of LLMs.<n>In this work, we reframe instruction hierarchy resolution as a reasoning task.
  arXiv  Detail & Related papers  (2025-10-30T22:13:31Z)
• TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree [52.44403214958304]
  In this paper, we introduce TreeLoRA, a novel approach that constructs layer-wise adapters by leveraging hierarchical gradient similarity.<n>To reduce the computational burden of task similarity estimation, we employ bandit techniques to develop an algorithm based on lower confidence bounds.<n> experiments on both vision transformers (ViTs) and large language models (LLMs) demonstrate the effectiveness and efficiency of our approach.
  arXiv  Detail & Related papers  (2025-06-12T05:25:35Z)
• Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines [71.14354526117958]
  In-context learning (ICL) is an important yet not fully understood ability of pre-trained large language models (LLMs)<n>We present LongGuide, which efficiently generates two parallel streams of guidelines capturing task language and format properties.<n>LongGuide automatically selects the best combination of guidelines, improving both strong open- and closed-source LLMs by over 5% in both zero- and few-shot settings.
  arXiv  Detail & Related papers  (2025-06-02T02:35:24Z)
• SMAC-R1: The Emergence of Intelligence in Decision-Making Tasks [8.05369449863086]
  We introduce SMAC-R1 which is based on the Qwen2.5-7B-Base LLM distilled from DeepSeek-Coder-v2.5-236B.<n>Similar to online reinforcement learning after behavior cloning in offline learning process, in our pipeline, agents leverage the DeepSeek LLM to generate decision tree code.<n>We conduct experiments in the original 23 SMAC tasks and 10 newly-designed tasks to demonstrate that our method can produce high-quality, interpretable decision trees.
  arXiv  Detail & Related papers  (2024-10-21T13:58:38Z)
• Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification [52.095460362197336]
  Large language models (LLMs) struggle with consistent and accurate reasoning. LLMs are trained primarily on correct solutions, reducing their ability to detect and learn from errors. We propose a novel collaborative method integrating Chain-of-Thought (CoT) and Program-of-Thought (PoT) solutions for verification.
  arXiv  Detail & Related papers  (2024-10-05T05:21:48Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• RNR: Teaching Large Language Models to Follow Roles and Rules [153.6596303205894]
  We propose model, an automated data generation pipeline that generates diverse roles and rules from existing IFT instructions. This data can then be used to train models that follow complex system prompts. Our framework significantly improves role and rule following capability in large language models.
  arXiv  Detail & Related papers  (2024-09-10T06:07:32Z)
• An Empirical Evaluation of Pre-trained Large Language Models for Repairing Declarative Formal Specifications [7.286515881369693]
  This paper systematically investigates the capacity of Large Language Models (LLMs) to repair declarative specifications in Alloy.<n>We designed 12 different repair settings, encompassing single-agent and dual-agent paradigms, utilizing various LLMs.<n>Our study reveals that dual-agent with auto-prompting setup outperforms the other settings, albeit with a marginal increase in the number of iterations and token usage.
  arXiv  Detail & Related papers  (2024-04-17T03:46:38Z)
• Text Generation with Efficient (Soft) Q-Learning [91.47743595382758]
  Reinforcement learning (RL) offers a more flexible solution by allowing users to plug in arbitrary task metrics as reward. We introduce a new RL formulation for text generation from the soft Q-learning perspective. We apply the approach to a wide range of tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
  arXiv  Detail & Related papers  (2021-06-14T18:48:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.