HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads

• URL: http://arxiv.org/abs/2604.17111v1
• Date: Sat, 18 Apr 2026 18:59:33 GMT
• Title: HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads
• Abstract summary: In a motivating incident, 3 of 11 parallel agents died from connection resets and HTTP 502 errors - a 27% failure rate.<n>We present HIVEMIND, a transparent HTTP proxy that applies five OS-inspired scheduling primitives to eliminate the failure modes caused by uncoordinated parallel execution.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When multiple LLM coding agents share a rate-limited API endpoint, they exhibit resource contention patterns analogous to unscheduled OS processes competing for CPU, memory, and I/O. In a motivating incident, 3 of 11 parallel agents died from connection resets and HTTP 502 errors - a 27% failure rate - despite the API having sufficient aggregate capacity to serve all 11 sequentially. We present HIVEMIND, a transparent HTTP proxy that applies five OS-inspired scheduling primitives - admission control, rate-limit tracking, AIMD backpressure with circuit breaking, token budget management, and priority queuing - to eliminate the failure modes caused by uncoordinated parallel execution. The proxy requires zero modifications to existing agent code and supports Anthropic, OpenAI, and local model APIs via auto-detected provider profiles. Our evaluation across seven scenarios (5-50 concurrent agents) shows that uncoordinated agents fail at 72-100% rates under contention, while HIVEMIND reduces failures to 0-18% and eliminates 48-100% of wasted compute. An ablation study reveals that transparent retry - not admission control - is the single most critical primitive, but the primitives are most effective in combination. Real-world validation against Ollama confirms that HIVEMIND adds under 3ms of proxy overhead per request. The system is open-source under the MIT license.

Related papers

• Where Is the Cost of Third-Party API Routers in Agentic Software Development? [16.886494606820495]
  Third-party API routers have become a common layer that unifies access across increasingly diverse LLM providers.<n>In coding-agents, high-autonomy operation is widely adopted because it reduces interaction overhead.<n>No mechanism verifies alignment between the provider's output and the repository-level actions ultimately executed by the agent.
  arXiv  Detail & Related papers  (2026-07-26T12:15:09Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents [61.184954205350984]
  ClawArena-Team is a benchmark of 41 multi-turn, multimodal, multi-agent scenarios spanning 258 evaluation rounds and 72 staged updates.<n>All scoring is execution-based with no LLM judge: an overall score -- the Subagent-Management Score (SMS) -- multiplies task correctness by a least-privilege and modality-routing factor.
  arXiv  Detail & Related papers  (2026-06-30T06:06:08Z)
• Verifier-First Evaluation of Agentic LLMs for Infrastructure-as-Code Generation [0.0]
  Infrastructure-as-Code (IaC) generation from natural language requires satisfying provider schemas, dependency planning, and organizational policy constraints.<n>We present a verifier-first empirical study of seven agentic strategies for Terraform generation evaluated on IaC-Eval v2, a modernized 186-task AWS/Terraform benchmark with Rego v1 policies.
  arXiv  Detail & Related papers  (2026-05-29T12:43:27Z)
• Deterministic Replay for AI Agent Systems [0.0]
  AI agent systems that couple large language models with external tools and APIs are inherently non-deterministic.<n>Existing observability platforms capture execution logs but cannot reproduce a run in isolation.<n>We present a developer-first CLI framework for deterministic replay of agent executions.
  arXiv  Detail & Related papers  (2026-04-30T12:00:45Z)
• FLARE: Agentic Coverage-Guided Fuzzing for LLM-Based Multi-Agent Systems [9.562231408217281]
  Multi-Agent LLM Systems (MAS) have been adopted to automate complex human tasks by breaking down tasks into subtasks.<n>MAS applications frequently encounter failures, including infinite loops and failed tool invocations.<n>Traditional software testing techniques are ineffective in detecting such failures due to the lack of LLM agent specification, the large behavioral space of MAS, and semantic-based correctness judgment.<n>This paper presents FLARE, a novel testing framework tailored for MAS. FLARE takes the source code of MAS as input and extracts specifications and behavioral spaces from agent definitions.
  arXiv  Detail & Related papers  (2026-04-07T00:47:37Z)
• AgentCgroup: Understanding and Controlling OS Resources of AI Agents [2.8139711959925244]
  AI agents are increasingly deployed in multi-tenant cloud environments, where they execute diverse tool calls within sandboxed containers.<n>We present a systematic characterization of OS-level resource dynamics in sandboxed AI coding agents.<n>Preliminary evaluation demonstrates improved multi-tenant isolation and reduced resource waste.
  arXiv  Detail & Related papers  (2026-02-10T02:37:42Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems [50.29939179830491]
  Failure attribution in LLM multi-agent systems remains underexplored and labor-intensive.<n>We develop and evaluate three automated failure attribution methods, summarizing their corresponding pros and cons.<n>The best method achieves 53.5% accuracy in identifying failure-responsible agents but only 14.2% in pinpointing failure steps.
  arXiv  Detail & Related papers  (2025-04-30T23:09:44Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.