Three Roles, One Model: Role Orchestration at Inference Time to Close the Performance Gap Between Small and Large Agents

• URL: http://arxiv.org/abs/2604.11465v2
• Date: Wed, 15 Apr 2026 13:28:59 GMT
• Title: Three Roles, One Model: Role Orchestration at Inference Time to Close the Performance Gap Between Small and Large Agents
• Abstract summary: We study whether inference-time scaffolding alone, without any additional training compute, can improve the performance of a small model in complex multi-step environments.<n>We evaluate Qwen3-8B on the AppWorld benchmark under both full-precision and 4-bit quantized configurations.<n>On full-precision inference, our scaffolded 8B model surpasses DeepSeek-Coder 33B Instruct (7.1%) from the original AppWorld evaluation.
• Score: 0.4666493857924357
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents show promise on realistic tool-use tasks, but deploying capable agents on modest hardware remains challenging. We study whether inference-time scaffolding alone, without any additional training compute, can improve the performance of a small model in complex multi-step environments. Operating on a single 24GB GPU, we evaluate Qwen3-8B on the AppWorld benchmark under both full-precision and 4-bit quantized configurations. Without any intervention, the raw model achieves just 5.4% (FP16) and 3.0% (AWQ) task goal completion. Guided by a systematic failure mode analysis, we introduce a three-tier inference scaffolding pipeline that deploys the same frozen model in three distinct roles: (1) a summarization model that preserves critical artifacts (tokens, credentials, API responses) while compressing dialogue history; (2) the main agent model that reasons over the compressed context; and (3) an isolated correction model that reviews and revises the agent's code output without access to conversation history, breaking repetitive failure loops. Applied to the same unmodified model, this scaffolding yields 8.9% (FP16) and 5.9% (AWQ) task goal completion, roughly doubling performance in both settings, with particularly strong gains on difficulty-1 tasks (15.8% to 26.3% FP16; 5.3% to 14.0% AWQ). On full-precision inference, our scaffolded 8B model surpasses DeepSeek-Coder 33B Instruct (7.1%) from the original AppWorld evaluation, demonstrating that structured inference-time interventions can make small models competitive with systems 4 times their size. We formalize the approach as a scaffolded policy over a frozen base model, three invocations of the same weights with different conditioning, drawing connections to test-time compute scaling and action-space shaping in reinforcement learning.

Related papers

• ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search [54.43696074323815]
  ZGCM-1 is a fully open 7B foundation model trained from scratch with extreme data, system, and algorithmic efficiency.<n>We establish an AI-native R&D workflow where agent swarms autonomously manage cluster operations.<n>ZGCM-1-7B is competitive across 7B model family on general benchmarks.
  arXiv  Detail & Related papers  (2026-09-11T17:18:04Z)
• AgentWeave: Routing Before Reasoning for Efficient Function Calling in Tool-Rich Language Models [0.0]
  We introduce AgentWeave, a deterministic pre-inference routing layer that constructs a bounded model-visible action space.<n>Relative to all-tools exposure, AgentWeave presents 70.18% fewer tools, uses 61.70% fewer input tokens, and exhibits 50.95% lower mean local-model latency.
  arXiv  Detail & Related papers  (2026-08-24T10:38:08Z)
• Compact SO(3) Equivariant Atomistic Foundation Models via Structural Pruning [4.793059213046564]
  We propose a structural pruning method for SO(3) equivariant atomistic foundation models.<n>The pruning is applied along the channel and order dimensions, with each irreducible representation kept or removed as a complete block.<n>The pruned MACE-MP model outperforms the official from-scratch trained small model on 7 of 9 metrics on the Matbench Discovery leaderboard.
  arXiv  Detail & Related papers  (2026-05-09T11:07:03Z)
• Scaling Trends for Multi-Hop Contextual Reasoning in Mid-Scale Language Models [0.0]
  We present a controlled study of multi-hop contextual reasoning in large language models.<n>We show that multi-agent systems show the inverse pattern, achieving up to 80% on reasoning tasks where rule-based methods fail.
  arXiv  Detail & Related papers  (2026-01-06T20:18:55Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models [14.479996384788445]
  We observe a puzzling phenomenon that larger models do emphNOT achieve better performance.<n>We propose a practical method to automatically identify and retain only these dominant layers.
  arXiv  Detail & Related papers  (2025-11-10T15:53:29Z)
• One Model to Critique Them All: Rewarding Agentic Tool-Use via Efficient Reasoning [54.580646706013965]
  Reward models (RMs) play a critical role in aligning large language models with human preferences.<n>We introduce ToolRM, a family of lightweight generative RMs tailored for general tool-use scenarios.<n>To build these models, we propose a novel pipeline that constructs pairwise preference data using rule-based scoring and multidimensional sampling.
  arXiv  Detail & Related papers  (2025-10-30T06:08:27Z)
• A2R: An Asymmetric Two-Stage Reasoning Framework for Parallel Reasoning [57.727084580884075]
  Asymmetric Two-Stage Reasoning framework designed to bridge gap between a model's potential and its actual performance.<n>A2R-Efficient is a "small-to-big" variant that combines a Qwen3-4B explorer with a Qwen3-8B synthesizer.<n>Results show A2R is not only a performance-boosting framework but also an efficient and practical solution for real-world applications.
  arXiv  Detail & Related papers  (2025-09-26T08:27:03Z)
• Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems [20.846301581161978]
  Failure attribution in multi-agent systems is a critical yet unsolved challenge.<n>Current methods treat this as a pattern recognition task over long conversation logs.<n>A2P Scaffolding transforms failure attribution from pattern recognition into a structured causal inference task.
  arXiv  Detail & Related papers  (2025-09-12T16:51:15Z)
• Skywork Open Reasoner 1 Technical Report [51.403686909760914]
  We present Skywork-OR1, an effective and scalable reinforcement learning (RL) implementation for long Chain-of-Thought (CoT) models.<n>Building on the DeepSeek-R1-Distill model series, our RL approach achieves notable performance gains.<n>Our Skywork-OR1-32B model surpasses both DeepSeek-R1 and Qwen3-32B on the AIME24 and AIME25 benchmarks.
  arXiv  Detail & Related papers  (2025-05-28T12:56:04Z)
• Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training [68.94373533768501]
  We model knowledge retention, the capacity of a pre-trained language model to memorize factual information from its corpus, and introduce a principled method to estimate it prior to training.<n>We propose Size-dependent Mutual Information (SMI), an information-theoretic predictor that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book question answering (QA) accuracy.
  arXiv  Detail & Related papers  (2025-02-06T13:23:53Z)
• Tool-Augmented Reward Modeling [58.381678612409]
  We propose a tool-augmented preference modeling approach, named Themis, to address limitations by empowering RMs with access to external environments. Our study delves into the integration of external tools into RMs, enabling them to interact with diverse external sources. In human evaluations, RLHF trained with Themis attains an average win rate of 32% when compared to baselines.
  arXiv  Detail & Related papers  (2023-10-02T09:47:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.