Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents

• URL: http://arxiv.org/abs/2608.21027v1
• Date: Fri, 21 Aug 2026 12:20:14 GMT
• Title: Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents
• Abstract summary: LLM agents are emerging as an important paradigm for real-world tasks that require reasoning, tool use, and sequential decision-making.<n>Existing approaches often rely on an expert solver or a critic that generates task-specific corrections.<n>We introduce Comparison-Only Tiny Advisor (COTA), a comparison-only framework for constructive runtime intervention.
• Score: 12.20001133898725
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: LLM agents are emerging as an important paradigm for real-world tasks that require reasoning, tool use, and sequential decision-making. As these agents operate over longer horizons, runtime intervention offers a way to improve reliability without retraining the underlying actor. Failure detection alone is insufficient. Effective intervention must also provide a useful direction for recovery. Existing approaches often rely on an expert solver or a critic that generates task-specific corrections, incurring either the cost of another capable solver or the capacity demands of a task-capable critic. We introduce Comparison-Only Tiny Advisor (COTA), a comparison-only framework for constructive runtime intervention. In COTA, a tiny comparator judges whether sampled alternatives lead to better continuations than the actor's proposal, and repeated comparisons determine when intervention is warranted. We train the comparator using pairwise supervision constructed from same-prefix counterfactual branches. Preferred alternatives are returned as non-binding advice, leaving the original actor to replan. Across WebShop, ALFWorld, and tau^3-Retail with three actors, COTA improves all nine evaluation settings and outperforms the compared baselines. These results show that constructive runtime intervention can remain effective even when the auxiliary model has substantially weaker task-solving capability than the actor.

Related papers

• S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement? [66.17176838165994]
  Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience.<n>It remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions.<n>We introduce textbfStextsuperscript3Gym, an interactive benchmark for evaluating LLM self-improvement.
  arXiv  Detail & Related papers  (2026-08-31T17:05:41Z)
• CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents [66.9575676266637]
  Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments.<n>In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and must be intercepted before execution.<n>We develop CAST, a critique-aware training framework that converts sparse task outcomes into action-level supervision for critique learning and policy optimization.
  arXiv  Detail & Related papers  (2026-08-31T01:59:51Z)
• CAFE: Self-Improving Search Agents Need Co-Evolving Feedback [64.5257526832476]
  Outcome-supervised search agents learn when and how to retrieve evidence, but terminal rewards neither localize intermediate errors nor redirect an ongoing trajectory before those errors compound.<n>Treating corrective feedback as a learned in-trajectory intervention couples the two roles: the agent must decide when to request and use feedback, while the critic must infer useful corrections from outcome-confounded rollouts whose failure patterns shift as the agent improves.
  arXiv  Detail & Related papers  (2026-08-25T16:39:00Z)
• CRISP: Critical Step Perception for Training Efficient Deep Search Agents [19.74419958232817]
  We propose CRISP, a framework for training efficient deep search agents through critical step perception.<n>Unlike prior efficiency methods that uniformly penalize tool use, CRISP distinguishes interactions that gather necessary evidence from redundant ones.<n>Experiments show that CRISP maintains competitive final-answer accuracy while reducing average interaction turns by 15.1% and 33.2%, respectively.
  arXiv  Detail & Related papers  (2026-08-03T08:15:18Z)
• Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference [51.55077364626896]
  We introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimize agent harness using only past trajectories.<n>RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel.<n>A single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading.
  arXiv  Detail & Related papers  (2026-06-04T09:26:00Z)
• Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents [6.158612515104146]
  We introduce Helpfulness-Harmfulness metrics to measure the tradeoff between agent feedback and reviewer feedback.<n>We evaluate our approach on BFCL and Tau2-Bench (multi-turn stateful scenarios), achieving +5.5% on irrelevance detection and +7.1% on multi-turn tasks.<n>Our metrics reveal that reviewer model choice is critical: the reasoning model o3-mini achieves a 3:1 benefit-to-risk ratio versus 2.1:1 for GPT-4o.
  arXiv  Detail & Related papers  (2026-04-29T22:09:47Z)
• Asymmetric Actor-Critic for Multi-turn LLM Agents [50.245019205783855]
  We propose an asymmetric actor-critic framework for reliable conversational agents.<n>A powerful proprietary LLM acts as the actor, while a smaller open-source critic provides runtime supervision.<n>We show that our approach significantly improves reliability and task success over strong single-agent baselines.
  arXiv  Detail & Related papers  (2026-03-31T22:56:21Z)
• Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs [57.10533368622962]
  Self-correction of large language models (LLMs) emerges as a critical component for enhancing their reasoning performance.<n>This study introduces CorrectBench, a benchmark developed to evaluate the effectiveness of self-correction strategies.<n>Our findings reveal that: 1) Self-correction methods can improve accuracy, especially for complex reasoning tasks; 2) Mixing different self-correction strategies yields further improvements, though it reduces efficiency; and 3) Reasoning LLMs (e.g., DeepSeek-R1) have limited optimization under additional self-correction methods and have high time costs.
  arXiv  Detail & Related papers  (2025-10-17T02:40:19Z)
• Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training [18.896813839389893]
  We propose an iterative self-training framework, Agent-R, that enables language Agent to Reflect on the fly.<n>Unlike traditional methods that reward or penalize actions based on correctness, Agent-R leverages MCTS to construct training data that recover correct trajectories from erroneous ones.<n>Our findings demonstrate that Agent-R continuously improves the model's ability to recover from errors and enables timely error correction.
  arXiv  Detail & Related papers  (2025-01-20T11:46:04Z)
• AgentRefine: Enhancing Agent Generalization through Refinement Tuning [28.24897427451803]
  Large Language Model (LLM) based agents have proved their ability to perform complex tasks like humans.<n>There is still a large gap between open-sourced LLMs and commercial models like the GPT series.<n>In this paper, we focus on improving the agent generalization capabilities of LLMs via instruction tuning.
  arXiv  Detail & Related papers  (2025-01-03T08:55:19Z)
• Learning to Use Tools via Cooperative and Interactive Agents [58.77710337157665]
  Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. We propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. Our experiments on three datasets show that the LLMs, when equipped with ConAgents, outperform baselines with substantial improvement.
  arXiv  Detail & Related papers  (2024-03-05T15:08:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.