AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents

• URL: http://arxiv.org/abs/2607.14547v1
• Date: Thu, 16 Jul 2026 04:07:22 GMT
• Title: AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents
• Abstract summary: AdaTurn is a budget-aware framework for visual agent training.<n>FA-DAPO converts an over-budget event into a trainable final-decision step.<n>AdaTurn substantially improves low-budget accuracy, for example raising VisualProbe-Medium from 36.7% to 47.6% at four turns.
• Score: 43.827500763103586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Active visual agents solve fine-grained image tasks by interleaving reasoning with image-grounding actions across multiple turns. However, deployment-time rollout budgets are rarely fixed: some requests permit long rollouts, while others require the agent to act under a tight turn limit. Existing methods train the policy as if the rollout budget were hidden, so when the available budget is smaller than the trajectory the agent prefers, the interaction is often truncated before any valid answer is produced; we term this failure \emph{catastrophic truncation}. To overcome this challenge, we present AdaTurn, a budget-aware framework that conditions the agent on the allowed number of turns and explicitly trains the boundary behavior induced by the budget. Our key component, Forced-Answer DAPO (FA-DAPO), converts the over-budget event from a masked or penalized failure into a trainable final-decision step, teaching the model to synthesize partial evidence when further tool use is no longer possible. We further randomize rollout budgets during both training and inference and introduce a load-balanced scheduler that makes such operations practical. AdaTurn substantially improves low-budget accuracy, for example raising VisualProbe-Medium from 36.7% to 47.6% at four turns, while preserving strong scaling at larger budgets and transferring effectively to multiple backbones and general multimodal benchmarks.

Related papers

• TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents [54.88208747581851]
  Training-free visual token pruning can reduce this cost, but cache reuse introduces a fundamental constraint.<n>We propose textbfmethod, a training-free framework for emphtextbfTrajectory-textbfrobust textbfAdmission and textbfCoverage-aware textbfEvidence ordering.
  arXiv  Detail & Related papers  (2026-09-09T15:12:48Z)
• TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents [51.523913302114266]
  TRACE (Turn-level Reward Assignment via Credit Estimation) is a dense credit-assignment method for agentic reinforcement learning.<n>It represents rollouts as state transitions at tool-call boundaries, obtains gold-answer log-probabilities from a frozen reference model, transforms them into log-ratio state values, and derives per-action rewards as Temporal-Difference changes in those values.<n>It substantially improves base-model tool-use ability using pure RL, without a cold-start supervised fine-tuning stage, an agentic mid-training stage, or training on live-web data.
  arXiv  Detail & Related papers  (2026-07-15T16:16:42Z)
• BAGEN: Are LLM Agents Budget-Aware? [35.04587529168086]
  Budget-Aware Agents (BAGEN) should treat budget as an active control signal, rather than a passive cost metric.<n>We first systematically define budget estimation as internal budgets (from agent computation) and external budgets (from agent actions)<n>We then formalize budget-awareness as progressive interval estimation.
  arXiv  Detail & Related papers  (2026-05-29T17:02:17Z)
• How Many Iterations to Jailbreak? Dynamic Budget Allocation for Multi-Turn LLM Evaluation [22.523809021772802]
  Key events in large language models (LLMs) in multi-turn conversational settings often emerge only after repeated interactions.<n>Recent conformal survival frameworks construct reliable lower predictive bounds (LPBs) on the number of iterations to trigger the event of interest.<n>We introduce DAPRO, the first theoretically valid dynamic budget allocation framework for bounding the time-to-event in multi-turn interactions.
  arXiv  Detail & Related papers  (2026-05-07T17:25:15Z)
• Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning [14.194667934432443]
  We formulate multi-turn reasoning as a sequential compute allocation problem and model it as a multi-objective Markov Decision Process.<n>We propose TAB: Turn-Adaptive Budgets, a budget allocation policy that learns to maximize task accuracy while respecting global per-problem token constraints.<n>For systems where a plan of all sub-questions is available apriori, we propose TAB All-SubQ, a budget allocation policy that budgets tokens based on the conversation history and all past and future sub-questions saving up to 40% tokens over baselines.
  arXiv  Detail & Related papers  (2026-04-06T20:48:51Z)
• Utility-Guided Agent Orchestration for Efficient LLM Tool Use [24.97962970486756]
  We study agent orchestration as an explicit decision problem.<n>We propose a utility-guided orchestration policy that selects among actions such as respond, retrieve, tool call, verify, and stop.
  arXiv  Detail & Related papers  (2026-03-20T12:29:12Z)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget [60.65072883773352]
  Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases.<n>We re-frame the budget setting problem as risk control, limiting the error rate while minimizing compute.<n>Our framework introduces an upper threshold that stops reasoning when the model is confident and a novel lower threshold that preemptively stops unsolvable instances.
  arXiv  Detail & Related papers  (2026-02-03T18:17:22Z)
• Budget-Aware Tool-Use Enables Effective Agent Scaling [82.6942342482552]
  Scaling test-time computation improves performance across different tasks on large language models (LLMs)<n>We study how to scale such agents effectively under explicit tool-call budgets, focusing on web search agents.<n>We introduce the Budget Tracker, a lightweight plug-in that provides the agent with continuous budget awareness.
  arXiv  Detail & Related papers  (2025-11-21T07:18:55Z)
• No-Regret Learning Under Adversarial Resource Constraints: A Spending Plan Is All You Need! [56.80767500991973]
  We focus on two canonical settings: $(i)$ online resource allocation where rewards and costs are observed before action selection, and $(ii)$ online learning with resource constraints where they are observed after action selection, under full feedback or bandit feedback.<n>It is well known that achieving sublinear regret in these settings is impossible when reward and cost distributions may change arbitrarily over time.<n>We design general (primal-)dual methods that achieve sublinear regret with respect to baselines that follow the spending plan. Crucially, the performance of our algorithms improves when the spending plan ensures a well-balanced distribution of the budget
  arXiv  Detail & Related papers  (2025-06-16T08:42:31Z)
• Frugal inference for control [2.20480252274709]
  Key challenge in advancing artificial intelligence is achieving the right balance between external movement and resource use.<n>We develop a version of the POMDP framework where the information gained through inference is treated as a resource that must be optimized alongside task performance and motion effort.<n>This work provides a foundation for a new type of rational computation that both brains and machines could use for effective but resource-efficient control under uncertainty.
  arXiv  Detail & Related papers  (2024-06-20T15:50:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.