From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification

• URL: http://arxiv.org/abs/2603.27299v1
• Date: Sat, 28 Mar 2026 15:04:31 GMT
• Title: From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification
• Abstract summary: The Semantic Router DSL is a non-Turing-complete policy language deployed in production for per-request inference.<n>This paper extends the same language from stateless, per-request routing to multi-step agent routing.
• Score: 10.651054435747385
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Semantic Router DSL is a non-Turing-complete policy language deployed in production for per-request LLM inference routing: content signals (embedding similarity, PII detection, jailbreak scoring) feed into weighted projections and priority-ordered decision trees that select a model, enforce privacy policies, and produce structured audit traces -- all from a single declarative source file. Prior work established conflict-free compilation for probabilistic predicates and positioned the DSL within the Workload-Router-Pool inference architecture. This paper extends the same language from stateless, per-request routing to multi-step agent workflows -- the full path from inference gateway to agent orchestration to infrastructure deployment. The DSL compiler emits verified decision nodes for orchestration frameworks (LangGraph, OpenClaw), Kubernetes artifacts (NetworkPolicy, Sandbox CRD, ConfigMap), YANG/NETCONF payloads, and protocol-boundary gates (MCP, A2A) -- all from the same source. Because the language is non-Turing-complete, the compiler guarantees exhaustive routing, conflict-free branching, referential integrity, and audit traces structurally coupled to the decision logic. Because signal definitions are shared across targets, a threshold change propagates from inference gateway to agent gate to infrastructure artifact in one compilation step -- eliminating cross-team coordination as the primary source of policy drift. We ground the approach in four pillars -- auditability, cost efficiency, verifiability, and tunability -- and identify the verification boundary at each layer.

Related papers

• SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations [88.71160380389277]
  We propose SW routers, a Similarity-Contractive Window Router for multi-turn large language model routing.<n> Experiments on multi-turn dialogue benchmarks demonstrate that SW consistently surpasses strong baselines.<n>Our results highlight that multi-turn large language model routing requires a joint design of context construction and evaluation.
  arXiv  Detail & Related papers  (2026-09-10T11:45:55Z)
• LMSM: LLM Security Framework Inspired by Linux Security Modules [56.93644694627783]
  Large language models (LLMs) are increasingly deployed with layered defenses, yet malicious prompts can still bypass them.<n>We present Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving.
  arXiv  Detail & Related papers  (2026-08-26T12:13:05Z)
• Formal Verification of Agentic Systems over Operational Data [59.98246281888422]
  Agentic systems driven by large language models (LLMs) are increasingly deployed in real-world where they act on persistent operational data.<n>Existing approaches do not provide such system-level guarantees.<n>We study here the verification of agentic systems comprising a single LLM and a tool orchestration harness over relational operational data.
  arXiv  Detail & Related papers  (2026-08-04T13:01:30Z)
• A Formal Hierarchical Architecture for Agentic Orchestration with Stack-Based Execution and Lazy Discovery [5.106901034158075]
  This paper presents a hierarchical, skill-based architecture for agentic orchestration.<n> Capabilities are organized as a rooted tree where internal nodes make routing decisions and leaf nodes execute deterministic tasks.<n>By replacing global memory with localized stack frames, the architecture prevents outputs from one execution branch from leaking into another.
  arXiv  Detail & Related papers  (2026-07-13T06:15:51Z)
• Agent-as-a-Router: Agentic Model Routing for Coding Tasks [54.00153674547733]
  Agent-as-a-router (AC) formalizes routing as a C-A-F loop (Context->Action->FeedbackContext)<n>AC achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic-programming tasks.
  arXiv  Detail & Related papers  (2026-06-22T06:37:31Z)
• The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project [30.96691028676722]
  vLLM Semantic Router project has released a series of work spanning signal-driven routing, context-length pool routing, router performance engineering, policy conflict detection, low-latency embedding models, category-aware semantic caching, user-feedback-driven routing adaptation, hallucination detection, and hierarchical content-safety classification for privacy and jailbreak protection.<n>This paper distills those results into the Workload-Pool-Pool architecture, a three-dimensional framework for LLM inference optimization.
  arXiv  Detail & Related papers  (2026-03-22T18:30:11Z)
• Conflict-Free Policy Languages for Probabilistic ML Predicates: A Framework and Case Study with the Semantic Router DSL [11.868735376461446]
  A class of routing and access-control systems base their decisions on probabilistic ML signals.<n>Two such signals, declared over categories the author intended to be disjoint, can both clear their thresholds on the same query and silently route it to the wrong model.<n>We show that for the embedding case, which dominates in practice, replacing independent thresholding with a temperature-scaled softmax partitions the embedding space into Voronoi regions where co-firing is impossible.
  arXiv  Detail & Related papers  (2026-03-18T18:14:34Z)
• A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governance [0.22940141855172028]
  This paper presents an assurance framework for Agentic AI systems using Large Language Models (LLMs)<n>Executions are instrumented as Message-Action Traces (MAT) with explicit step and trace contracts.<n>The framework includes stress testing, formulated as a budgeted counterexample search over bounded perturbations.
  arXiv  Detail & Related papers  (2026-03-18T10:23:48Z)
• vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models [8.433829083279518]
  vLLM Semantic Router is a signal-driven decision routing framework for Mixture-of-Modality (MoM) model deployments.<n>The system extracts heterogeneous signal types from each request.<n>Different deployment scenarios are expressed as different signal-decision configurations over the same architecture.
  arXiv  Detail & Related papers  (2026-02-23T15:00:01Z)
• Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem [90.17610617854247]
  We introduce the Agentic Learning Ecosystem (ALE), a foundational infrastructure that optimize the production pipeline for agentic model.<n>ALE consists of three components: ROLL, a post-training framework for weight optimization; ROCK, a sandbox environment manager for trajectory generation; and iFlow CLI, an agent framework for efficient context engineering.<n>We release ROME, an open-source agent grounded by ALE and trained on over one million trajectories.
  arXiv  Detail & Related papers  (2025-12-31T14:03:39Z)
• AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering [51.07491603393163]
  tAgent is a framework that formulates multi-agent QA as a knowledge-graph-guided routing problem supervised by empirical performance signals.<n>By leveraging soft supervision and weighted aggregation of agent outputs, Agent learns principled collaboration schemes that capture the complementary strengths of diverse agents.
  arXiv  Detail & Related papers  (2025-10-06T23:20:49Z)
• Rethinking Testing for LLM Applications: Characteristics, Challenges, and a Lightweight Interaction Protocol [83.83217247686402]
  Large Language Models (LLMs) have evolved from simple text generators into complex software systems that integrate retrieval augmentation, tool invocation, and multi-turn interactions.<n>Their inherent non-determinism, dynamism, and context dependence pose fundamental challenges for quality assurance.<n>This paper decomposes LLM applications into a three-layer architecture: textbftextitSystem Shell Layer, textbftextitPrompt Orchestration Layer, and textbftextitLLM Inference Core.
  arXiv  Detail & Related papers  (2025-08-28T13:00:28Z)
• RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory [57.449129198822476]
  RCR is a role-aware context routing framework for multi-agent large language model (LLM) systems.<n>It dynamically selects semantically relevant memory subsets for each agent based on its role and task stage.<n>A lightweight scoring policy guides memory selection, and agent outputs are integrated into a shared memory store.
  arXiv  Detail & Related papers  (2025-08-06T21:59:34Z)
• HADA: Human-AI Agent Decision Alignment Architecture [0.0]
  HADA is a protocol- and framework reference architecture that keeps both large language model (LLM) agents and legacy algorithms aligned with organizational targets and values.<n>Technical and non-technical actors can query, steer, audit, or contest every decision across strategic, tactical, and real-time horizons.
  arXiv  Detail & Related papers  (2025-06-01T14:04:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.