R2V Agent: Teaching SLMs When to Ask for Help

• URL: http://arxiv.org/abs/2605.16604v1
• Date: Fri, 15 May 2026 20:10:24 GMT
• Title: R2V Agent: Teaching SLMs When to Ask for Help
• Abstract summary: We introduce textbfR2V-Agent, a risk-calibrated SLM-LLM routing framework for interactive agents.<n>R2V combines four components: a distilled small language model (SLM) policy, a stronger teacher LLM, a lightweight process verifier that scores candidate actions at each step, and a calibrated step-level router.
• Score: 13.959667066774124
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient agentic systems should incur expensive frontier-model costs only on decisions where a cheaper local model is likely to fail. Existing LLM cascades usually route whole queries before execution, but task difficulty shifts mid-trajectory - after flaky tool calls, truncated observations, or compounding local errors - making pre-execution routing brittle. We introduce \textbf{R2V-Agent}, a risk-calibrated SLM-LLM routing framework for interactive agents. R2V combines four components: a distilled small language model (SLM) policy, a stronger teacher LLM, a lightweight process verifier that scores candidate actions at each step, and a calibrated step-level router. The router is our central contribution: after the SLM is trained, it estimates residual failure risk at each step and escalates only when teacher intervention is warranted. To make the routing problem well-defined, we first train a stable local SLM using a standard offline pipeline: behavioral cloning (BC) on teacher trajectories, followed by verifier-guided Direct Preference Optimization (DPO) with consistency regularization. The router is then trained on this fixed policy's residual failures using Brier-calibrated probability estimation and a Conditional Value-at-Risk (CVaR)-constrained objective that penalizes worst-case failures across perturbation seeds. Across HumanEval+, TextWorld, and TerminalBench with four SLM backbones, R2V improves the reliability-cost frontier: it achieves $94.3\%$ HumanEval+ success with $0.60\%$ LLM escalation, recovers TextWorld from $64.6\%$ SLM-only success to $98.2\%$ at $41.7\%$ escalation, and reaches $93.3\%$ TerminalBench success at $33.9\%$ LLM calls, roughly half the heuristic-router cost.

Related papers

• TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision [2.681967953218225]
  Agents with smaller language-model backbones can drift into persistent failure modes, whereas those with larger backbones are generally more reliable but more costly.<n>This reliability-cost trade-off motivates routing methods that decide when to invoke an agent with a larger backbone.<n>Our method, TACIT-SWITCH, learns permanent handoff policies from accumulated trajectory evidence and Teacher-Annotated Censored Intervention Times.
  arXiv  Detail & Related papers  (2026-08-28T04:33:11Z)
• LLM Readiness Harness: Evaluation, Observability, and CI Gates for LLM/RAG Applications [51.56484100374058]
  We present a readiness harness for LLM and RAG applications that turns evaluation into a deployment decision workflow.<n>The system combines automated benchmarks, OpenTelemetry observability, and CI quality gates under a minimal API contract.<n>We evaluate the harness on ticket-routing and BEIR tasks with full Azure matrix coverage.
  arXiv  Detail & Related papers  (2026-03-28T18:03:32Z)
• Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints [18.52522897906341]
  We study the problem of routing queries to large language models (LLMs)<n>We propose a batch-level, resource-aware routing framework that jointly optimize assignment for each batch.<n> Experiments on two multi-task benchmarks show that improves accuracy by 1-14% on non-robust counterparts.
  arXiv  Detail & Related papers  (2026-03-25T22:24:11Z)
• Graph-Based Self-Healing Tool Routing for Cost-Efficient LLM Agents [0.0]
  Self-Healing Router is a fault-tolerant orchestration architecture.<n>It treats most agent control-flow decisions as routing rather than reasoning.<n>Every failure is either a logged reroute or an explicit escalation, never a silent skip.
  arXiv  Detail & Related papers  (2026-03-02T07:21:15Z)
• Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA [16.758340727602793]
  Table Question Answering (TQA) aims to answer natural language questions over structured tables.<n>Large Language Models (LLMs) enable promising solutions to this problem, with operator-centric solutions that generate table manipulation pipelines in a multi-step manner offering state-of-the-art performance.<n>We propose Operation-R1, the first framework that trains lightweight LLMs via a novel variant of reinforcement learning with verifiable rewards to produce high-quality data-preparation pipelines for TQA in a single inference step.
  arXiv  Detail & Related papers  (2026-02-26T07:49:50Z)
• R2-Router: A New Paradigm for LLM Routing with Reasoning [58.929817721828194]
  We show that R2- achieves state-of-the-art performance at 4-5x lower cost compared with existing routers.<n>This work opens a new direction: routing as reasoning, where routers evolve from reactive selectors to deliberate reasoners.
  arXiv  Detail & Related papers  (2026-02-02T21:23:51Z)
• Pay for Hints, Not Answers: LLM Shepherding for Cost-Efficient Inference [7.865726406769634]
  Small Language Models (SLMs) offer dramatic cost savings yet lag substantially in accuracy.<n>We introduce LLM Shepherding, a framework that requests only a short prefix (a hint) from the LLM and provides it to SLM.<n>Shepherding generalizes both routing and cascading, and it achieves lower cost under oracle decision-making.
  arXiv  Detail & Related papers  (2026-01-29T18:52:54Z)
• Dr.LLM: Dynamic Layer Routing in LLMs [55.11953638340419]
  Dr.LLM is a retrofittable framework that equips pretrained models with lightweight per-layer routers deciding to skip, execute, or repeat a block.<n>On ARC (logic) and DART (math), Dr.LLM improves accuracy by up to +3.4%p while saving 5 layers per example on average.
  arXiv  Detail & Related papers  (2025-10-14T17:51:26Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• Towards Efficient Automatic Self-Pruning of Large Language Models [55.90119819642064]
  Post-training structured pruning is a promising solution that prunes Large Language Models without the need for retraining.<n>We argue that the key to mitigating this issue lies in accurately determining the pruning rate for each layer.<n>We introduce $textbfSelf-Pruner$ an end-to-end automatic self-pruning framework for LLMs, which efficiently search layer-wise pruning rates.
  arXiv  Detail & Related papers  (2025-02-20T09:59:50Z)
• MasRouter: Learning to Route LLMs for Multi-Agent Systems [14.029698552632107]
  Multi-agent systems powered by Large Language Models (LLMs) have been demonstrated to push the boundaries of LLM capabilities.<n>Current routing methods effectively reduce overhead in single-agent scenarios by customizing LLM selection for each query.<n>We first introduce the problem of Multi-Agent Routing System (MASR), which integrates all components of MAS into a unified routing framework.<n>Mas is (1) high-performing, achieving a $1.8%sim8.2%$ improvement over the state-of-the-art method on MBPP; (2) economical, reducing overhead by up to $52.07%$ compared to S
  arXiv  Detail & Related papers  (2025-02-16T14:00:59Z)
• Universal Model Routing for Efficient LLM Inference [69.86195589350264]
  Model routing is a technique for reducing the inference cost of large language models (LLMs)<n>We propose UniRoute, a new approach to the problem of dynamic routing, where new, previously unobserved LLMs are available at test time.<n>We show that these are estimates of a theoretically optimal routing rule, and quantify their errors via an excess risk bound.
  arXiv  Detail & Related papers  (2025-02-12T20:30:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.