SPOQ: Specialist Orchestrated Queuing for Multi-Agent Software Engineering

• URL: http://arxiv.org/abs/2606.03115v1
• Date: Tue, 02 Jun 2026 03:59:56 GMT
• Title: SPOQ: Specialist Orchestrated Queuing for Multi-Agent Software Engineering
• Abstract summary: Multi-agent AI systems show promise for automating software engineering tasks.<n>Existing approaches suffer from coordination overhead, quality control gaps, and limited human oversight.<n>We introduce SPOQ (Specialist Orchestrated Queuing), a methodology combining three innovations.
• Score: 2.0769172070951067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent AI systems show promise for automating software engineering tasks, yet existing approaches suffer from coordination overhead, quality control gaps, and limited human oversight. We introduce SPOQ (Specialist Orchestrated Queuing), a methodology combining three innovations: (1) wave-based topological dispatch that computes parallel execution waves from task dependency graphs; (2) dual validation gates applying quality metrics before execution (planning validation) and after (code validation) to reduce rework cycles; and (3) Human-as-an-Agent (HaaA) integration, where a human specialist participates in decomposition and can be consulted during execution. SPOQ uses a three-tier agent hierarchy (Opus workers, Sonnet reviewers, Haiku investigators) to optimize cost-quality tradeoffs. We evaluate SPOQ through four experiments. Experiment 1: wave dispatch approaches the critical-path lower bound (ratio 1.03--1.11, speedup up to 14.3x); on a 2-slot local backend it delivers a stable 1.4x speedup. Experiment 2: SPOQ improves planning coverage from 93.0 to 99.75, eliminates cyclic plans, and lifts parallelism from 31.0 to 75.25. Experiment 3: dual validation reduces defects from 0.34 to 0.20 per task and lifts test pass rate from 91.25% to 99.75%. Experiment 4: human review reduces residual defects from 0.47 to 0.03 per task. Results are replicated on a locally hosted open-weights model (Qwen3.6-35B-A3B), verifying gains are attributable to orchestration rather than any specific model. A longitudinal study across 17 repositories, 8,589 commits, 1,822 tasks, and 13,866 tests (99.87% pass rate) provides ecological validation.

Related papers

• Recursive Synthesis for Long-Horizon Terminal Tasks [50.39352383646813]
  Recursive Synthetic Terminal Tasks (RST) is a framework for constructing long-horizon terminal-agent tasks at scale.<n>RST produces 37,484 synthesized terminal-agent tasks at roughly $0.05 per task.<n>Fine-tuning on trajectories improves Qwen3.5-27B and Qwen3.5-122B-A10B by up to 10 points on Terminal-Bench 2, Terminal-Bench Hard, and Long-Horizon Terminal Bench.
  arXiv  Detail & Related papers  (2026-08-05T23:24:26Z)
• Determinants and Limits of LLM Security-Tool Orchestration: A Study with HexStrike-AI [2.2616066835313253]
  We run 86 picoCTF challenges across seven categories and three difficulty tiers, under three tool-access regimes and three model/client configurations.<n>We apply corrections to existing tools, agent-behavior changes, and eleven new capability tools, and re-run the previously-unsuccessful trials.<n>The overall solve rate rises from 55.4% to 72.0%, and every configuration improves significantly.
  arXiv  Detail & Related papers  (2026-07-03T02:20:04Z)
• To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair [18.653708704742353]
  Executions can be time-consuming and expensive, yet their impact on program repair agents remains underexplored.<n>Code execution is used across all agents and models analyzed, with an average of 8.8 test runs per task.<n>Current agents apply execution indiscriminately, paying its cost on instances where it provides little benefit.
  arXiv  Detail & Related papers  (2026-06-25T12:49:59Z)
• Learning the ARTS of Search for Automated Discovery [62.89970162621959]
  We propose Agentic Reasoning for Tree Search (ARTS) to navigate scientific discovery.<n>We deploy a reasoning language model to navigate this space.<n>We show that ARTS outperforms leading algorithms.
  arXiv  Detail & Related papers  (2026-06-20T05:47:37Z)
• A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback [4.062253229926774]
  A-ProS is an autonomous AI agent that solves competitive programming problems through a hybrid multi-model feedback framework.<n>A-ProS achieves over 2x greater gains compared to baseline agent loops.
  arXiv  Detail & Related papers  (2026-05-18T08:55:30Z)
• EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents [75.01735520608075]
  Existing benchmarks mostly test literal Theory of Mind (ToM) by asking direct belief questions.<n>We introduce EnactToM, an evolving benchmark of 300 embodied multi-agent tasks set in a 3D household.<n>On the hard split, all seven evaluated frontier models score 0.0% Pass3 on functional task completion, while averaging 45.0% on literal belief probes.
  arXiv  Detail & Related papers  (2026-05-11T00:04:19Z)
• AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking [13.891522069967507]
  We present AgentEval, a framework that formalizes agent executions as evaluation directed acyclic graphs (DAGs)<n>AgentEval achieves 2.17x higher failure detection recall than end-to-end evaluation and 72% root cause accuracy against an 81% human ceiling.
  arXiv  Detail & Related papers  (2026-04-26T07:38:47Z)
• Claw-Eval: Toward Trustworthy Evaluation of Autonomous Agents [66.97968363332465]
  We introduce Claw-Eval, an end-to-end evaluation suite addressing all three gaps in agent benchmarks.<n>Claw-Eval comprises 300 human-verified tasks spanning 9 categories across three groups.<n>Every agent action is recorded through three independent evidence channels.
  arXiv  Detail & Related papers  (2026-04-07T17:43:18Z)
• AIRA_2: Overcoming Bottlenecks in AI Research Agents [31.269750755536247]
  Existing research has identified three structural performance bottlenecks in AI research agents.<n>We introduce AIRA$$, which addresses these bottlenecks through three architectural choices.<n>On MLE-bench-30, AIRA$$ achieves a mean Percentile Rank of 71.8% at 24 hours - surpassing the previous best of 69.9%.
  arXiv  Detail & Related papers  (2026-03-27T15:02:43Z)
• Orchestrating Human-AI Software Delivery: A Retrospective Longitudinal Field Study of Three Software Modernization Programs [0.0]
  We report a longitudinal field study of Chiron, an industrial platform that coordinates humans and AI agents across four delivery stages.<n>The study covers three real software modernization programs -- a banking migration (30k LOC), a large accounting modernization (400k LOC) and a.NET/Angular mortgage modernization (30k LOC)<n>The benchmark separates observed outcomes (stage durations, task volumes, validation-stage issues, first-release coverage) from modeled outcomes.
  arXiv  Detail & Related papers  (2026-03-20T15:14:36Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.