Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks

• URL: http://arxiv.org/abs/2608.25399v1
• Date: Wed, 26 Aug 2026 05:58:12 GMT
• Title: Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks
• Abstract summary: We study the effects of different task specifications on agentic token spend with the Kimi K3 model at three thinking efforts.<n>We show that reducing a full task specification to a bare user story raises token spend by $29.7%$, while run-to-run variance remains unaffected by any prompt changes.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic coding workflows are now widely deployed in real-world systems. With long-horizon reasoning and tool use, token usage has become an important consideration for both cost and efficiency. Two engineers using AI will solve the same problem differently. How the specification of a task shapes an agent's token spend, and whether that spend can be predicted in advance, are open questions. Here, we study the effects of different task specifications on agentic token spend with the Kimi K3 model at three thinking efforts. Across $2,700$ runs, we show that reducing a full task specification to a bare user story raises token spend by $29.7\%$, while run-to-run variance remains unaffected by any prompt changes. We show that prompt-sensitivity is task-dependent, running from $13\%$ to $115\%$. We fit a simple predictor that can price a full distribution of task specifications and thinking effort configurations from a single cheap probe on an unseen task within $36\%$, improving over prior work in predicting token spend. Our work provides initial results quantifying the effects of task specification on agentic token spend and introduces a method that can be used to systematically evaluate the cost of AI coding workflows.

Related papers

• Putting Registers to Work: Task Registers for Token Pruning in Vision Transformers [5.410304374735817]
  Token-pruning policies are usually designed for a single recognition pipeline, but pretrained Vision Transformers are reused across tasks with different spatial demands.<n>We ask which parts of a pruning policy transfer across image classification, semantic segmentation, and object detection.<n>For each pipeline, controlled probes freeze the no-pruning checkpoint and apply a series of parameter-free reduction criteria at one eligible layer at a time without retraining.<n>The probes reveal three differences: segmentation and detection rank the criteria differently, classification is especially sensitive to attention-based pruning in the earliest layers, and the dense tasks prefer opposite recovery endpoints.
  arXiv  Detail & Related papers  (2026-08-11T14:43:52Z)
• Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution [1.733151600403503]
  Large language model (LLM) agents increasingly automate multi-step engineering and informatics.<n>They often follow a maximum-context-first strategy--re-reading files and dependencies they have already seen--turning a one-line edit into a small code-base audit.<n>We argue the missing capability is task-aware execution-scope estimation.
  arXiv  Detail & Related papers  (2026-07-14T17:59:31Z)
• Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents [34.54961677428331]
  We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget.<n>We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps.<n>Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor.
  arXiv  Detail & Related papers  (2026-06-12T23:30:14Z)
• Not All Errors Are Equal: Consequence-Aware Reasoning Compute Allocation [5.523004155648451]
  We propose consequence-aware test-time compute allocation.<n>We conduct experiments on SWE-bench Lite and evaluate cross-dataset behavior on Multi-SWE-bench mini.
  arXiv  Detail & Related papers  (2026-06-03T03:29:57Z)
• Scaling Laws for Agent Harnesses via Effective Feedback Compute [53.68149869349268]
  emphEffective Feedback Compute (EFC) is a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for subsequent decisions.<n>EFC-based coordinates consistently predict failure rates better than raw-compute baselines.
  arXiv  Detail & Related papers  (2026-05-28T09:45:47Z)
• How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks [40.99169244450745]
  We present the first systematic study of token consumption patterns in agentic coding tasks.<n>We evaluate models' ability to predict their own token costs before task execution.<n>Our study offers new insights into the economics of AI agents and can inspire future research.
  arXiv  Detail & Related papers  (2026-04-24T17:54:47Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• Algorithm Design for Continual Learning in IoT Networks [16.35495567193046]
  Continual learning (CL) is a new online learning technique over sequentially generated streaming data from different tasks.<n>In practical IoT networks, an autonomous vehicle to sample data and learn different tasks can route and alter the order of task pattern.
  arXiv  Detail & Related papers  (2024-12-22T02:36:09Z)
• Label Budget Allocation in Multi-Task Learning [61.12008399759143]
  In multi-task learning, related tasks provide information to each other and improve overall performance. How should the label budget be allocated among different tasks to achieve optimal multi-task performance? We are the first to propose and formally define the label budget allocation problem in multi-task learning.
  arXiv  Detail & Related papers  (2023-08-24T17:38:14Z)
• Efficient Controllable Multi-Task Architectures [85.76598445904374]
  We propose a multi-task model consisting of a shared encoder and task-specific decoders where both encoder and decoder channel widths are slimmable. Our key idea is to control the task importance by varying the capacities of task-specific decoders, while controlling the total computational cost. This improves overall accuracy by allowing a stronger encoder for a given budget, increases control over computational cost, and delivers high-quality slimmed sub-architectures.
  arXiv  Detail & Related papers  (2023-08-22T19:09:56Z)
• FAMO: Fast Adaptive Multitask Optimization [48.59232177073481]
  We introduce Fast Adaptive Multitask Optimization FAMO, a dynamic weighting method that decreases task losses in a balanced way. Our results indicate that FAMO achieves comparable or superior performance to state-of-the-art gradient manipulation techniques.
  arXiv  Detail & Related papers  (2023-06-06T15:39:54Z)
• Collaborative Anomaly Detection [66.51075412012581]
  We propose collaborative anomaly detection (CAD) to jointly learn all tasks with an embedding encoding correlations among tasks. We explore CAD with conditional density estimation and conditional likelihood ratio estimation. It is beneficial to select a small number of tasks in advance to learn a task embedding model, and then use it to warm-start all task embeddings.
  arXiv  Detail & Related papers  (2022-09-20T18:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.