Agentic AI Systems Should Be Designed as Marginal Token Allocators

• URL: http://arxiv.org/abs/2605.01214v1
• Date: Sat, 02 May 2026 03:06:02 GMT
• Title: Agentic AI Systems Should Be Designed as Marginal Token Allocators
• Abstract summary: We argue that agentic AI systems should be designed and evaluated as emphmarginal token allocation economies.<n>We show that all four layers are solving the emphsame first-order condition.
• Score: 2.629665653283008
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This position paper argues that agentic AI systems should be designed and evaluated as \emph{marginal token allocation economies} rather than as text generators priced by the unit. We follow a single request -- a developer asking a coding agent to fix a failing test -- through four economic layers that today are designed in isolation: a router that decides which model answers, an agent that decides whether to plan, act, verify, or defer, a serving stack that decides how to produce each token, and a training pipeline that decides whether the trace is worth learning from. We show that all four layers are solving the \emph{same} first-order condition -- marginal benefit equals marginal cost plus latency cost plus risk cost -- with different index sets and different prices. The framing is deliberately minimal: we do not propose a complete theory of AI economics. But adopting marginal token allocation as the shared accounting object explains why systems that locally minimize tokens globally misallocate them, predicts a small set of recurring failure modes (over-routing, over-delegation, under-verification, serving congestion, stale rollouts, cache misuse), and points to a concrete research agenda in token-aware evaluation, autonomy pricing, congestion-priced serving, and risk-adjusted RL budgeting.

Related papers

• Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation [64.51088225051959]
  Heterogeneous AI systems can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost.<n>We formalize this tradeoff as an instance of Pandora's Box, the classical problem of optimal search with costly inspection.<n>Under a Gaussian signal model, the resulting policies have closed-form value-of-information expressions that determine, for each specialist and input, whether refining the value estimate is worth its cost.
  arXiv  Detail & Related papers  (2026-08-20T17:54:37Z)
• Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing [5.00725149774706]
  Enterprise AI coding assistants incur substantial inference spend, and naive token-cost minimization often fails to reduce end-to-end cost once retries, escalations, and developer wait time are included.<n>We present Task-to-Model Optimization (T2MO), a data-driven methodology for optimizing model selection in production coding.<n>We treat each developer session as a task that can be discovered, classified, graded for difficulty, benchmarked in a production-like harness, and routed to the cheapest model able to complete it within quality and latency constraints.
  arXiv  Detail & Related papers  (2026-08-09T06:57:53Z)
• Instance-Optimal Estimation with Multiple LLM Judges on a Budget [84.31744861038106]
  We formalize this question as *budgeted heteroskedastic multi-judge estimation*.<n>Given $K$ prompt-response pairs, $J$ judges with known costs, and unknown query-judge variances, the goal is to estimate a bounded score vector while minimizing an $ell_p$-error.<n>We prove that EST-IVWE matches the oracle IVWE rate up to lower-order terms in the budget.
  arXiv  Detail & Related papers  (2026-05-22T08:26:08Z)
• Token Economics for LLM Agents: A Dual-View Study from Computing and Economics [25.319434762851156]
  This survey presents the first comprehensive survey of Token Economics.<n>We conceptualize tokens as production factors, exchange mediums, and units of account.<n>We lay the theoretical foundation for scalable next-generation agent systems.
  arXiv  Detail & Related papers  (2026-05-09T18:18:51Z)
• Avoiding Overthinking and Underthinking: Curriculum-Aware Budget Scheduling for LLMs [1.7499351967216341]
  Budget-Adaptive Curriculum Reasoning (BCAE) is a unified framework that jointly optimize reasoning quality and token efficiency.<n>BCAE includes a emphbudget-conditioned unified policy, a emphcurriculum-aware budget scheduler, and a emphtruncation-aware dense reward mechanism.
  arXiv  Detail & Related papers  (2026-03-29T18:31:09Z)
• Real-Time AI Service Economy: A Framework for Agentic Computing Across the Continuum [10.35088918434398]
  This article shows that the structure of service-dependency graphs, modelled as DAGs, is a primary determinant of whether decentralised, price-based resource allocation can work reliably at scale.<n>We propose a hybrid management architecture in which cross-domain sub-graphs into resource slices that present a simpler, well-structured interface to the rest of the market.
  arXiv  Detail & Related papers  (2026-03-05T19:14:54Z)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget [60.65072883773352]
  Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases.<n>We re-frame the budget setting problem as risk control, limiting the error rate while minimizing compute.<n>Our framework introduces an upper threshold that stops reasoning when the model is confident and a novel lower threshold that preemptively stops unsolvable instances.
  arXiv  Detail & Related papers  (2026-02-03T18:17:22Z)
• Market-Driven Subset Selection for Budgeted Training [1.7969777786551429]
  We propose a market-based framework that treats each training example as a tradeable contract.<n>On GSM8K mathematical reasoning under strict 60k-token budgets, our selector achieves parity with strong single-signal baselines.<n>Our framework unifies multi-signal data curation under fixed computational budgets for prompt-level reasoning and classification tasks.
  arXiv  Detail & Related papers  (2025-10-02T18:12:03Z)
• Probably Anytime-Safe Stochastic Combinatorial Semi-Bandits [81.60136088841948]
  We propose an algorithm that minimizes the regret over the horizon of time $T$. The proposed algorithm is applicable to domains such as recommendation systems and transportation.
  arXiv  Detail & Related papers  (2023-01-31T03:49:00Z)
• Quantization for decentralized learning under subspace constraints [61.59416703323886]
  We consider decentralized optimization problems where agents have individual cost functions to minimize subject to subspace constraints. We propose and study an adaptive decentralized strategy where the agents employ differential randomized quantizers to compress their estimates. The analysis shows that, under some general conditions on the quantization noise, the strategy is stable both in terms of mean-square error and average bit rate.
  arXiv  Detail & Related papers  (2022-09-16T09:38:38Z)
• Inducing Equilibria via Incentives: Simultaneous Design-and-Play Finds Global Optima [114.31577038081026]
  We propose an efficient method that tackles the designer's and agents' problems simultaneously in a single loop. Although the designer does not solve the equilibrium problem repeatedly, it can anticipate the overall influence of the incentives on the agents. We prove that the algorithm converges to the global optima at a sublinear rate for a broad class of games.
  arXiv  Detail & Related papers  (2021-10-04T06:53:59Z)
• VCG Mechanism Design with Unknown Agent Values under Stochastic Bandit Feedback [104.06766271716774]
  We study a multi-round welfare-maximising mechanism design problem in instances where agents do not know their values. We first define three notions of regret for the welfare, the individual utilities of each agent and that of the mechanism. Our framework also provides flexibility to control the pricing scheme so as to trade-off between the agent and seller regrets.
  arXiv  Detail & Related papers  (2020-04-19T18:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.