How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks

• URL: http://arxiv.org/abs/2604.22750v2
• Date: Wed, 29 Apr 2026 17:20:11 GMT
• Title: How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
• Abstract summary: We present the first systematic study of token consumption patterns in agentic coding tasks.<n>We evaluate models' ability to predict their own token costs before task execution.<n>Our study offers new insights into the economics of AI agents and can inspire future research.
• Score: 40.99169244450745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The wide adoption of AI agents in complex human workflows is driving rapid growth in LLM token consumption. When agents are deployed on tasks that require a significant amount of tokens, three questions naturally arise: (1) Where do AI agents spend the tokens? (2) Which models are more token-efficient? and (3) Can agents predict their token usage before task execution? In this paper, we present the first systematic study of token consumption patterns in agentic coding tasks. We analyze trajectories from eight frontier LLMs on SWE-bench Verified and evaluate models' ability to predict their own token costs before task execution. We find that: (1) agentic tasks are uniquely expensive, consuming 1000x more tokens than code reasoning and code chat, with input tokens rather than output tokens driving the overall cost; (2) token usage is highly variable and inherently stochastic: runs on the same task can differ by up to 30x in total tokens, and higher token usage does not translate into higher accuracy; instead, accuracy often peaks at intermediate cost and saturates at higher costs; (3) models vary substantially in token efficiency: on the same tasks, Kimi-K2 and Claude-Sonnet-4.5, on average, consume over 1.5 million more tokens than GPT-5; (4) task difficulty rated by human experts only weakly aligns with actual token costs, revealing a fundamental gap between human-perceived complexity and the computational effort agents actually expend; and (5) frontier models fail to accurately predict their own token usage (with weak-to-moderate correlations, up to 0.39) and systematically underestimate real token costs. Our study offers new insights into the economics of AI agents and can inspire future research in this direction.

Related papers

• Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks [0.0]
  We study the effects of different task specifications on agentic token spend with the Kimi K3 model at three thinking efforts.<n>We show that reducing a full task specification to a bare user story raises token spend by $29.7%$, while run-to-run variance remains unaffected by any prompt changes.
  arXiv  Detail & Related papers  (2026-08-26T05:58:12Z)
• Not All LLM Reasoning is Visible in the Chain-of-Thought [48.129884329641975]
  frontier models exhibit invisible reasoning by leveraging semantically irrelevant filler tokens to improve performance on synthetic reasoning tasks.<n>We show that filler tokens enable Claude Opus 4.5 to satisfy a hidden modular arithmetic constraint without sacrificing accuracy on its primary task.
  arXiv  Detail & Related papers  (2026-07-24T21:32:48Z)
• Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents [34.54961677428331]
  We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget.<n>We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps.<n>Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor.
  arXiv  Detail & Related papers  (2026-06-12T23:30:14Z)
• How Long Is a Piece of String? A Brief Empirical Analysis of Tokenizers [39.60188078597529]
  tokenization varies significantly across models and domains of text, making naive interpretation of token counts problematic.<n>Our analysis challenges commonly held intuitions about token lengths, finding them to be overly simplistic.
  arXiv  Detail & Related papers  (2026-01-16T18:58:29Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• MARCOS: Deep Thinking by Markov Chain of Continuous Thoughts [82.46857666702924]
  We present a new paradigm for reasoning in large language models (LLMs)<n>Instead of autoregressively generating tokens, we model reasoning as a hidden Markov chain of continuous, high-dimensional "thoughts"<n>For the first time, MARCOS achieves performance comparable to token-based CoT, even surpassing it by 4.7% on GSM8K with up to 15.7x speedup in inference.
  arXiv  Detail & Related papers  (2025-09-29T16:44:22Z)
• Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model [7.8354921036790275]
  Large Reasoning Models (LRMs) excel at solving complex problems but face an overthinking dilemma.<n>When handling simple tasks, they often produce verbose responses overloaded with thinking tokens.<n>These tokens trigger unnecessary high-level reasoning behaviors like reflection and backtracking, reducing efficiency.
  arXiv  Detail & Related papers  (2025-06-30T13:30:33Z)
• IGD: Token Decisiveness Modeling via Information Gain in LLMs for Personalized Recommendation [79.22388408461458]
  We introduce an Information Gain-based Decisiveness-aware Token handling (IGD) strategy that integrates token decisiveness into both tuning and decoding.<n>IGD consistently improves recommendation accuracy, achieving significant gains on widely used ranking metrics compared to strong baselines.
  arXiv  Detail & Related papers  (2025-06-16T08:28:19Z)
• Multipole Attention for Efficient Long Context Reasoning [64.94673641704289]
  Large Reasoning Models (LRMs) have shown promising accuracy improvements on complex problem-solving tasks.<n>LRMs need to generate long chain-of-thought reasoning in order to think before answering.<n>We introduce Multipole Attention, which accelerates autoregressive reasoning by only computing exact attention for the most important tokens.
  arXiv  Detail & Related papers  (2025-06-16T03:00:40Z)
• TokenButler: Token Importance is Predictable [8.514853311344458]
  Large Language Models (LLMs) rely on the Key-Value (KV) Cache to store token history, enabling efficient decoding of tokens.<n>Prior research has shown that only a small subset of tokens contribute meaningfully to each decoding step.<n>We introduce TokenButler, a high-granularity, query-aware predictor that learns to identify these critical tokens.
  arXiv  Detail & Related papers  (2025-03-10T16:41:14Z)
• Let's Think Dot by Dot: Hidden Computation in Transformer Language Models [30.972412126012884]
  Chain-of-thought responses from language models improve performance across most benchmarks. We show that transformers can use meaningless filler tokens in place of a chain of thought to solve two hard algorithmic tasks. We find that learning to use filler tokens is difficult and requires specific, dense supervision to converge.
  arXiv  Detail & Related papers  (2024-04-24T09:30:00Z)
• Revisiting Token Pruning for Object Detection and Instance Segmentation [25.3324628669201]
  We investigate token pruning to accelerate inference for object and instance segmentation. We show a reduction in performance decline from 1.5 mAP to 0.3 mAP in both boxes and masks, compared to existing token pruning methods.
  arXiv  Detail & Related papers  (2023-06-12T11:55:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.