Runtime Burden Allocation for Structured LLM Routing in Agentic Expert Systems: A Full-Factorial Cross-Backend Methodology

• URL: http://arxiv.org/abs/2604.01235v1
• Date: Thu, 26 Mar 2026 18:50:46 GMT
• Title: Runtime Burden Allocation for Structured LLM Routing in Agentic Expert Systems: A Full-Factorial Cross-Backend Methodology
• Abstract summary: As large language models (LLMs) become core control components in agentic AI systems, reliable structured routing must balance correctness, latency, and implementation cost.<n>We show that this balance is shaped not only by prompts or schemas, but also by how structural work is allocated across the generation stack.<n>We evaluate this formulation through a comprehensive full-factorial benchmark covering 48 deployment configurations and 15,552 requests across OpenAI, Gemini, and Llama backends.
• Score: 16.053669481561354
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured LLM routing is often treated as a prompt-engineering problem. We argue that it is, more fundamentally, a systems-level burden-allocation problem. As large language models (LLMs) become core control components in agentic AI systems, reliable structured routing must balance correctness, latency, and implementation cost under real deployment constraints. We show that this balance is shaped not only by prompts or schemas, but also by how structural work is allocated across the generation stack: whether output structure is emitted directly by the model, compressed during transport, or reconstructed locally after generation. We evaluate this formulation through a comprehensive full-factorial benchmark covering 48 deployment configurations and 15,552 requests across OpenAI, Gemini, and Llama backends. Our central finding is consequential: there is no universal best routing mode. Instead, backend-specific interaction effects dominate performance. Modes that remain highly reliable on Gemini and OpenAI can suffer substantial correctness degradation on Llama, while efficiency gains from compressed realization are strongly backend-dependent. Rather than presenting another isolated model comparison, this work contributes a deployable framework for reasoning about structured routing under heterogeneous backend conditions. We provide a cross-backend evaluation methodology and practical deployment guidance for navigating the correctness-cost-latency frontier in production-grade agentic expert systems.

Related papers

• Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers [72.66457738838515]
  We propose a Transformer-based Fixed-Point Reasoning Model that uses fixed-point convergence as an end-to-end halting mechanism in a looped architecture.<n>FPRM is effective on common reasoning benchmarks, namely Sudoku, Maze, state-tracking, and ARC-AGI.
  arXiv  Detail & Related papers  (2026-06-16T17:36:33Z)
• Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrieval [0.0]
  Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding their responses in external knowledge.<n>This paper presents an Agent-Orchestrated Adaptive RAG framework that introduces dynamic query decomposition, iterative retrieval, and a bounded self-reflective evaluation loop.
  arXiv  Detail & Related papers  (2026-06-04T03:38:46Z)
• Constraint Decay: The Fragility of LLM Agents in Backend Code Generation [9.659020624935687]
  Large Language Model (LLM) agents demonstrate strong performance in autonomous code generation under loose specifications.<n>Production-grade software requires strict adherence to structural constraints, such as architectural patterns, databases, and object-relational mappings.<n>We present a systematic study evaluating how well agents handle structural constraints in backend generation.
  arXiv  Detail & Related papers  (2026-05-07T15:44:40Z)
• One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement [37.27419953581617]
  Large Language Models (LLMs) often fail to utilize their latent reasoning capabilities.<n>We propose ReQueR, a modular framework that treats reasoning elicitation as an inference-time alignment task.
  arXiv  Detail & Related papers  (2026-04-28T09:52:21Z)
• Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimization [58.59491516762626]
  We propose AMRO-S, an efficient and interpretable routing framework for Multi-Agent Systems (MAS)<n>AMRO-S models MAS routing as a semantic-conditioned path selection problem, enhancing routing performance through three key mechanisms.<n>Extensive experiments on five public benchmarks and high-concurrency stress tests demonstrate that AMRO-S consistently improves the quality--cost trade-off over strong routing baselines.
  arXiv  Detail & Related papers  (2026-03-13T12:26:05Z)
• Relatron: Automating Relational Machine Learning over Relational Databases [50.94254514286021]
  We present a study that unifies RDL and DFS in a shared design space and conducts architecture-centric searches across diverse RDB tasks.<n>Our analysis yields three key findings: (1) RDL does not consistently outperform DFS, with performance being highly task-dependent; (2) no single architecture dominates across tasks, underscoring the need for task-aware model selection; and accuracy is an unreliable guide for choice architecture.
  arXiv  Detail & Related papers  (2026-02-26T02:45:22Z)
• Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration [56.074760766965085]
  PRISM achieves a dynamics-aware framework that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge.<n>Our findings suggest that disentangling data based on internal optimization regimes is crucial for scalable and robust agent alignment.
  arXiv  Detail & Related papers  (2026-01-12T05:43:20Z)
• Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem [90.17610617854247]
  We introduce the Agentic Learning Ecosystem (ALE), a foundational infrastructure that optimize the production pipeline for agentic model.<n>ALE consists of three components: ROLL, a post-training framework for weight optimization; ROCK, a sandbox environment manager for trajectory generation; and iFlow CLI, an agent framework for efficient context engineering.<n>We release ROME, an open-source agent grounded by ALE and trained on over one million trajectories.
  arXiv  Detail & Related papers  (2025-12-31T14:03:39Z)
• Higress-RAG: A Holistic Optimization Framework for Enterprise Retrieval-Augmented Generation via Dual Hybrid Retrieval, Adaptive Routing, and CRAG [0.0]
  Higress RAG MCP Server is a novel, enterprise-centric architecture for AI deployment.<n>System orchestrates Adaptive Routing, Semantic Caching, Hybrid Retrieval, and Corrective RAG.<n>System offers a scalable, hallucination-resistant solution for enterprise AI deployment.
  arXiv  Detail & Related papers  (2025-12-30T05:28:05Z)
• AgentBalance: Backbone-then-Topology Design for Cost-Effective Multi-Agent Systems under Budget Constraints [7.38359558170225]
  Large Language Model (LLM)-based multi-agent systems (MAS) are becoming indispensable building blocks for web-scale applications.<n>We present AgentBalance, a framework for constructing cost-effective MAS under explicit token-cost and latency budgets.
  arXiv  Detail & Related papers  (2025-12-12T10:08:03Z)
• SCoTER: Structured Chain-of-Thought Transfer for Enhanced Recommendation [24.019381388104236]
  We propose SCoTER, a unified framework that treats pattern discovery and structure-aware transfer as a jointly optimized problem.<n>Specifically, SCoTER operationalizes this through two synergistic components: a GVM pipeline for automated pattern discovery and a structure-preserving integration architecture that transfers stepwise logic to efficient models.
  arXiv  Detail & Related papers  (2025-11-24T03:00:04Z)
• Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generation [55.47971671635531]
  Large Language Models (LLMs) have shown remarkable performance on general Question Answering (QA)<n>Retrieval-Augmented Generation (RAG) addresses this limitation by enriching LLMs with external knowledge.<n>Existing systems primarily rely on unstructured documents, while largely overlooking relational databases.
  arXiv  Detail & Related papers  (2025-09-30T22:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.