Don't Overthink It: Inter-Rollout Action Agreement as a Free Adaptive-Compute Signal for LLM Agents

• URL: http://arxiv.org/abs/2604.08369v1
• Date: Thu, 09 Apr 2026 15:34:22 GMT
• Title: Don't Overthink It: Inter-Rollout Action Agreement as a Free Adaptive-Compute Signal for LLM Agents
• Abstract summary: Inference-time compute scaling has emerged as a powerful technique for improving the reliability of large language model (LLM) agents.<n>We introduce TrACE, a training-free controller that allocates LLM calls adaptively across agent timesteps by measuring inter-rollout action agreement.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inference-time compute scaling has emerged as a powerful technique for improving the reliability of large language model (LLM) agents, but existing methods apply compute uniformly: every decision step receives the same budget regardless of its difficulty. We introduce TrACE (Trajectorical Adaptive Compute via agrEement), a training-free controller that allocates LLM calls adaptively across agent timesteps by measuring inter-rollout action agreement. At each step, TrACE samples a small set of candidate next actions and measures how consistently the model commits to the same action. High agreement signals an easy decision; the controller commits immediately. Low agreement signals uncertainty; the controller samples additional rollouts up to a configurable cap before committing to the plurality action. No learned components, no external verifier, and no human labels are required. We evaluate TrACE against greedy decoding and fixed-budget self-consistency (SC-4, SC-8) on two benchmarks spanning single-step reasoning (GSM8K, n=50) and multi-step household navigation (MiniHouse, n=30), using a Qwen 2.5 3B Instruct model running on CPU. TrACE-4 matches SC-4 accuracy while using 33% fewer LLM calls on GSM8K and 39% fewer on MiniHouse. TrACE-8 matches SC-8 accuracy with 55% fewer calls on GSM8K and 65% fewer on MiniHouse. We further show that inter-rollout agreement is a reliable signal of step-level success, validating the core hypothesis that the model's own output consistency encodes difficulty information that can be exploited without training. TrACE is the first training-free, per-timestep adaptive-compute controller for LLM agents to be evaluated on multi-step sequential decision tasks.

Related papers

• AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning [10.408251801653723]
  Large language models (LLMs) achieve strong reasoning performance, which depends critically on time decisions.<n>Recent adaptive methods partially address this limitation, but they primarily adapt either recommended decodingity (how the model explores) or reasoning compute (how long the model reasons) in isolation.<n>We introduce a within-trajectory joint control view, and instantiate it in AutoCRAT.
  arXiv  Detail & Related papers  (2026-08-30T19:24:58Z)
• Closed-Loop Control with Rule-Aligned Small Language Models and Multi-Agent Self-Correction [17.527470001317685]
  This work investigates whether a compact Small Language Model (SLM) can be retrained for control reasoning and embedded in a validator-guided correction loop.<n>In randomized thermal-control simulations (30 experiments with 500 steps each), the framework achieves 91.5% average action-alignment accuracy.<n>Under symbolic re-mapping, it maintains a 95% in-range rate, indicating robust physical regulation despite reduced token-level agreement.
  arXiv  Detail & Related papers  (2026-06-24T13:49:01Z)
• Agent Explorative Policy Optimization for Multimodal Agentic Reasoning [97.64835302176056]
  Agentic reasoning interleaves two behaviors with a structural asymmetry: thinking and tool use.<n>Under standard RL recipes like GRPO, the gap manifests as two diagnostic symptoms during training.<n>We propose AXPO (Agent eXplorative Policy Optimization) for each all-wrong tool-using subgroup.
  arXiv  Detail & Related papers  (2026-05-27T17:36:39Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Minimal Information Control Invariance via Vector Quantization [2.094349987888854]
  We study how many distinct control signals are needed to render a compact set forward invariant under sampled-data control.<n>We propose a vector-quantized autoencoder that jointly learns a state-space partition and a finite control codebook.<n>We empirically characterize the minimum sensing resolution compatible with safe operation.
  arXiv  Detail & Related papers  (2026-04-03T15:53:07Z)
• ACAR: Adaptive Complexity Routing for Multi-Model Ensembles with Auditable Decision Traces [3.151184728006369]
  We present ACAR, a measurement framework for studying multi-model orchestration under auditable conditions.<n>ACAR uses self-consistency variance (sigma) computed from N=3 probe samples to route tasks across single-model, two-model, and three-model execution modes.<n>We evaluate ACAR on 1,510 tasks spanning four benchmarks, producing more than 7,550 auditable runs.
  arXiv  Detail & Related papers  (2026-02-06T23:27:17Z)
• Adaptive Reinforcement and Model Predictive Control Switching for Safe Human-Robot Cooperative Navigation [10.614812216110673]
  This paper addresses the challenge of human-guided navigation for mobile collaborative robots under simultaneous proximity regulation and safety constraints.<n>We introduce Adaptive Reinforcement and Model Predictive Control Switching (ARMS), a hybrid learning-control framework.<n>We show that ARMS achieves an 82.5 percent success rate in highly cluttered environments.
  arXiv  Detail & Related papers  (2026-01-23T12:02:18Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression [68.69801176669843]
  We propose an online post-training RL method that prunes redundant steps and estimates difficulty.<n> TRAAC (Think Right with Adaptive, Attentive Compression) achieves an average absolute accuracy gain of 8.4%.<n>Although our models are trained on math datasets, they show accuracy and efficiency gains on out-of-distribution non-math datasets.
  arXiv  Detail & Related papers  (2025-10-02T02:00:20Z)
• Intra-request branch orchestration for efficient LLM reasoning [52.68946975865865]
  Large Language Models (LLMs) increasingly rely on inference-time reasoning algorithms to improve accuracy on complex tasks.<n>Prior work has largely focused on reducing token usage, often at the expense of accuracy, while overlooking other latency factors.<n>We present DUCHESS, an LLM serving system that reduces cost and latency without sacrificing accuracy through intra-request branch orchestration guided by predictions.
  arXiv  Detail & Related papers  (2025-09-29T15:52:08Z)
• AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security [74.22452069013289]
  AegisLLM is a cooperative multi-agent defense against adversarial attacks and information leakage.<n>We show that scaling agentic reasoning system at test-time substantially enhances robustness without compromising model utility.<n> Comprehensive evaluations across key threat scenarios, including unlearning and jailbreaking, demonstrate the effectiveness of AegisLLM.
  arXiv  Detail & Related papers  (2025-04-29T17:36:05Z)
• Actor-Critic based Improper Reinforcement Learning [61.430513757337486]
  We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process. We propose two algorithms: (1) a Policy Gradient-based approach; and (2) an algorithm that can switch between a simple Actor-Critic scheme and a Natural Actor-Critic scheme.
  arXiv  Detail & Related papers  (2022-07-19T05:55:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.