Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

• URL: http://arxiv.org/abs/2607.28573v1
• Date: Thu, 30 Jul 2026 17:36:36 GMT
• Title: Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs
• Abstract summary: We present a systematic empirical study of inference-time scaling in local computer-use agents.<n>Our results show that additional computation often yields diminishing returns while changing failure modes.<n>Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks.
• Score: 7.85315290601208
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deploying autonomous computer-use agents (CUAs) locally is increasingly important for privacy, cost efficiency, and practical usability, yet improving their performance under strict hardware constraints remains challenging. While recent studies show that inference-time scaling can improve frontier computer-use agents through additional computation during execution, its effectiveness for resource-constrained local models remains poorly understood. We present a systematic empirical study of inference-time scaling in local CUAs across contextual, temporal, structural, and parallel dimensions. We evaluate Qwen3-VL-8B/30B-A3B, UI-TARS-1.5-7B, and OpenCUA-7B on the OSWorld benchmark. Our results show that additional computation often yields diminishing returns while changing failure modes. Contextual scaling provides historical grounding that improves trajectory stability and task accuracy, but its gains saturate as token cost increases and failures shift from repetitive or stalled trajectories toward premature false successes. Temporal scaling similarly reduces max-step stalls, yet does not substantially improve task success, indicating that longer horizons often extend erroneous trajectories rather than correct them. We further find that structural decomposition can introduce planning and formatting overhead in local two-stage agents, while parallel scaling partially mitigates these failures at a substantial computational cost. Overall, our findings suggest that efficient local CUAs require selective compute allocation, failure-aware control mechanisms, and agentic frameworks designed around the capabilities and limitations of local models.

Related papers

• DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners? [57.585275546688116]
  Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents.<n>We argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world.<n>We introduce DIRECT, a routing framework that uses multimodal scene context to allocate compute per prompt.
  arXiv  Detail & Related papers  (2026-06-10T17:58:49Z)
• C-Phase-Aware Compilation for Efficient Fault-Tolerant Quantum Execution [1.635939418192339]
  This work presents a microarchitecture-aware compilation approach that integrates algorithmic structure directly with lattice surgery execution.<n>We design a dynamic, event-driven scheduling strategy that accurately models spatial layout and routing constraints.<n>This approach substantially reduces idle resources and achieves up to a 59.7$times$ reduction in execution time compared to standard baselines.
  arXiv  Detail & Related papers  (2026-05-13T19:03:18Z)
• How LLMs Fail and Generalize in RTL Coding for Hardware Design? [56.361436215029045]
  We introduce a new error taxonomy grounded in problem solvability, inspired by cognitive theory.<n>Our taxonomy categorizes failures into syntactic, semantic, solvable functional, and unsolvable functional types.
  arXiv  Detail & Related papers  (2026-04-26T14:34:49Z)
• Generative Control as Optimization: Time Unconditional Flow Matching for Adaptive and Robust Robotic Control [33.03635235189535]
  We introduce Generative Control as Optimization (GeCO), a time-unconditional framework that transforms action synthesis into iterative optimization.<n>Test-time inference becomes an adaptive process that allocates computation based on convergence-exiting early for simple states while refining longer for difficult ones.<n>We validate GeCO on standard simulation benchmarks and demonstrate seamless scaling to pi0-series Vision-Language-Action (VLA) models.
  arXiv  Detail & Related papers  (2026-03-18T15:27:17Z)
• AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting [59.31340724915079]
  Event Spotting is a key task for applications in sports analytics, robotics, and autonomous systems.<n>bfAdaSpot achieves state-of-the-art performance under strict evaluation metrics.
  arXiv  Detail & Related papers  (2026-02-25T16:24:48Z)
• Learning from Yesterday's Error: An Efficient Online Learning Method for Traffic Demand Prediction [6.104967994062357]
  FORESEE (Forecasting Online with Residual Smoothing and Ensemble Experts) is a lightweight online adaptation framework that is accurate, robust, and efficient.<n>It corrects today's forecast in each region using yesterday's prediction error, through exponential smoothing guided by a mixture-of-experts mechanism.<n>Experiments on seven real-world datasets with backbone models demonstrate that FORESEE consistently improves prediction accuracy, maintains robustness even when distribution shifts are minimal.
  arXiv  Detail & Related papers  (2026-02-25T10:19:39Z)
• When Learning Hurts: Fixed-Pole RNN for Real-Time Online Training [58.25341036646294]
  We analytically examine why learning recurrent poles does not provide tangible benefits in data and empirically offer real-time learning scenarios.<n>We show that fixed-pole networks achieve superior performance with lower training complexity, making them more suitable for online real-time tasks.
  arXiv  Detail & Related papers  (2026-02-25T00:15:13Z)
• R-ConstraintBench: Evaluating LLMs on NP-Complete Scheduling [0.0]
  We present R-ConstraintBench, a framework that evaluates models on Resource-Constrained Project Scheduling Problems (RCPSP)<n>We instantiate the benchmark in a data center migration setting and evaluate multiple LLMs using feasibility and error analysis.<n> Empirically, strong models are near-ceiling on precedence-only DAGs, but feasibility performance collapses when downtime, temporal windows, and disjunctive constraints interact.
  arXiv  Detail & Related papers  (2025-08-21T03:35:58Z)
• Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute [61.00662702026523]
  We propose a unified Test-Time Compute scaling framework that leverages increased inference-time instead of larger models.<n>Our framework incorporates two complementary strategies: internal TTC and external TTC.<n>We demonstrate our textbf32B model achieves a 46% issue resolution rate, surpassing significantly larger models such as DeepSeek R1 671B and OpenAI o1.
  arXiv  Detail & Related papers  (2025-03-31T07:31:32Z)
• Faster Diffusion Action Segmentation [9.868244939496678]
  Temporal Action Classification (TAS) is an essential task in video analysis, aiming to segment and classify continuous frames into distinct action segments. Recent advances in diffusion models have demonstrated substantial success in TAS tasks due to their stable training process and high-quality generation capabilities. We propose EffiDiffAct, an efficient and high-performance TAS algorithm.
  arXiv  Detail & Related papers  (2024-08-04T13:23:18Z)
• FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge Computing Migrations [54.34189781923818]
  FIRE is a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment.<n>We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function.<n>We show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.
  arXiv  Detail & Related papers  (2022-09-28T19:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.