Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

• URL: http://arxiv.org/abs/2606.15017v1
• Date: Fri, 12 Jun 2026 23:30:14 GMT
• Title: Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
• Abstract summary: We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget.<n>We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps.<n>Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor.
• Score: 34.54961677428331
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume test-time tokens, a cost rarely reported alongside the actor's inference cost. We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget. We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps. Across three WebArena domains and three models, Gemini 3 Flash, GPT-5.4-mini, and Qwen 3.6-27B, the vanilla baseline matches or surpasses all three augmentation methods in aggregate success rate while often using fewer total tokens. We observe a similar trend on WorkArena-L1 with Qwen 3.6-27B, indicating that the effect extends to enterprise knowledge-work tasks. Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor. We further show that run-to-run variance materially affects outcomes and should be reported as a core evaluation criterion for online web agents.

Related papers

• Can your AI agent be cheaper? Investigating the effects of task specifications on token spend in agentic coding tasks [0.0]
  We study the effects of different task specifications on agentic token spend with the Kimi K3 model at three thinking efforts.<n>We show that reducing a full task specification to a bare user story raises token spend by $29.7%$, while run-to-run variance remains unaffected by any prompt changes.
  arXiv  Detail & Related papers  (2026-08-26T05:58:12Z)
• TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents [51.523913302114266]
  TRACE (Turn-level Reward Assignment via Credit Estimation) is a dense credit-assignment method for agentic reinforcement learning.<n>It represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values.<n>It substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data.
  arXiv  Detail & Related papers  (2026-07-15T16:16:42Z)
• PANDO: Efficient Multimodal AI Agents via Online Skill Distillation [8.784808015938543]
  We introduce PANDO, a single-rollout online skill-distillation framework.<n>On the full set of 910 VisualWebArena tasks, PANDO achieves a 58.3% success rate.<n>A 300-task ablation shows that rules and routines provide most of the success gains.
  arXiv  Detail & Related papers  (2026-05-24T00:07:25Z)
• Counterfactual Trace Auditing of LLM Agent Skills [38.396785087675774]
  We introduce a framework for measuring how a skill changes agent behavior.<n>We instantiate CTA on SWE-Skills-Bench with Claude across 49 software engineering tasks.<n>Pass rate changes by only +0.3 percentage points on average, suggesting little aggregate effect.
  arXiv  Detail & Related papers  (2026-05-12T10:56:18Z)
• How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks [40.99169244450745]
  We present the first systematic study of token consumption patterns in agentic coding tasks.<n>We evaluate models' ability to predict their own token costs before task execution.<n>Our study offers new insights into the economics of AI agents and can inspire future research.
  arXiv  Detail & Related papers  (2026-04-24T17:54:47Z)
• Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets [14.028807574426748]
  We present an information-theoretic argument, grounded in the Data Processing Inequality.<n>We find that SAS consistently match or outperform MAS on multi-hop reasoning tasks when reasoning tokens are held constant.<n>Overall, our results suggest that, for multi-hop reasoning tasks, many reported advantages of multi-agent systems are better explained by unaccounted computation and context effects.
  arXiv  Detail & Related papers  (2026-04-02T18:47:48Z)
• Learning Query-Aware Budget-Tier Routing for Runtime Agent Memory [56.0946692457838]
  BudgetMem is a runtime agent memory framework for explicit, query-aware performance-cost control.<n>A lightweight router performs budget-tier routing across modules to balance task performance and memory construction cost.<n>Across LoCoMo, LongMemEval, and HotpotQA, BudgetMem surpasses strong baselines when performance is prioritized.
  arXiv  Detail & Related papers  (2026-02-05T18:57:09Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Reward [85.84943447589511]
  This paper introduces AC3 (Actor-Critic for Continuous Chunks), a novel RL framework that learns to generate high-dimensional, continuous action sequences.<n>To make this learning process stable and data-efficient, AC3 incorporates targeted stabilization mechanisms for both the actor and the critic.
  arXiv  Detail & Related papers  (2025-08-15T01:27:15Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning [102.36450942613091]
  We propose an inverse reinforcement learning algorithm, called emphinverse temporal difference learning (ITD) We show how to seamlessly integrate ITD with learning from online environment interactions, arriving at a novel algorithm for reinforcement learning with demonstrations, called $Psi Phi$-learning.
  arXiv  Detail & Related papers  (2021-02-24T21:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.