Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents

• URL: http://arxiv.org/abs/2604.06753v1
• Date: Wed, 08 Apr 2026 07:20:23 GMT
• Title: Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents
• Abstract summary: We compare six inference-time paradigms, namely Direct, CoT, ReAct, Plan-Execute, Reflection, and ReCode.<n>We find that reasoning structure helps dramatically on some tasks but hurts on others.<n>We propose a select-then-solve approach: before answering each task, a lightweight embedding-based router selects the most suitable paradigm.
• Score: 40.71556008019808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When an LLM-based agent improves on a task, is the gain from the model itself or from the reasoning paradigm wrapped around it? We study this question by comparing six inference-time paradigms, namely Direct, CoT, ReAct, Plan-Execute, Reflection, and ReCode, across four frontier LLMs and ten benchmarks, yielding roughly 18,000 runs. We find that reasoning structure helps dramatically on some tasks but hurts on others: ReAct improves over Direct by 44pp on GAIA, while CoT degrades performance by 15pp on HumanEval. No single paradigm dominates, and oracle per-task selection beats the best fixed paradigm by 17.1pp on average. Motivated by this complementarity, we propose a select-then-solve approach: before answering each task, a lightweight embedding-based router selects the most suitable paradigm. Across four models, the router improves average accuracy from 47.6% to 53.1%, outperforming the best fixed paradigm at 50.3% by 2.8pp and recovering up to 37% of the oracle gap. In contrast, zero-shot self-routing only works for GPT-5 at 67.1% and fails for weaker models, all trailing the learned router. Our results argue that reasoning paradigm selection should be a per-task decision made by a learned router, not a fixed architectural choice.

Related papers

• LLMs Can Predict Failure Risk, But Struggle to Predict Which Collaboration Protocol Pays Off: Cost-Aware Protocol Routing Across Reasoning Tasks [3.789882665982407]
  Multi-agent large language model (LLM) systems can improve reasoning by spending more computation.<n>We isolate this decision by running every problem under four protocols while holding the solver fixed within each setting.
  arXiv  Detail & Related papers  (2026-08-14T22:35:37Z)
• FLARE: Few-shot Learning-based Adaptive Reflective Engine [0.9932714016119233]
  We introduce FLARE (Few-shot Learning-based Adaptive Reflective Engine), a framework that leverages advanced reflective mechanisms to optimize instructions.<n>We evaluate our method across a diverse suite of benchmarks using the GPT-5 series of models.<n>Our results demonstrate that FLARE consistently outperforms GEPA, winning on every task-model pair.
  arXiv  Detail & Related papers  (2026-08-03T22:13:12Z)
• Influence of Prompt Engineering on Small Language Models for Guarded Query Routing [0.0]
  We investigate whether compact open-weight Small Language Models (SLMs) can jointly handle both tasks under latency constraints.<n>We find that mid-scale SLMs come close to frontier model routing quality at much lower latency.<n>Results show that prompt optimization techniques enable SLMs to handle such cases gracefully, without changing the models' weights.
  arXiv  Detail & Related papers  (2026-07-06T09:21:18Z)
• optimize_anything: A Universal API for Optimizing any Text Parameter [98.42497715725356]
  We show a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs.<n>We show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm.
  arXiv  Detail & Related papers  (2026-05-19T10:18:12Z)
• ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference [60.958331943869126]
  ODAR-Expert is an adaptive routing framework that optimize the accuracy-efficiency trade-off via principled resource allocation.<n>We show strong and consistent gains, including 98.2% accuracy on MATH and 54.8% on Humanity's Last Exam.
  arXiv  Detail & Related papers  (2026-02-27T05:22:01Z)
• MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning [85.05204262206296]
  Large reasoning models (LRMs) achieve strong performance by producing long chains of thought, but their inference costs are high.<n>Small language models (SLMs) are far more efficient, yet struggle on multi-step reasoning tasks.<n>We propose MentorCollab, an inference-time collaboration method in which an LRM selectively and sparsely guides an SLM, rather than taking over generation.
  arXiv  Detail & Related papers  (2026-02-05T04:58:16Z)
• You Only Need Your Transformer 25% of the Time: Meaning-First Execution for Eliminating Unnecessary Inference [0.0]
  We introduce Meaning-First Execution (MFEE), a control-plane architecture implementing this framework.<n>MFEE achieves 78.1% execution reduction while maintaining 100% exact-match equivalence for invoked executions.
  arXiv  Detail & Related papers  (2025-12-29T08:03:52Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• TrajSelector: Harnessing Latent Representations for Efficient and Effective Best-of-N in Large Reasoning Model [21.82904448561646]
  Large language models (LLMs) have shown remarkable progress in complex reasoning tasks.<n>Best-of-N selection paradigm yields scalable performance improvements by selecting from multiple independently generated reasoning trajectories.<n>We introduce TrajSelector, an efficient and effective Best-of-N framework that exploit the hidden states in the sampler LLM for process-level scoring.
  arXiv  Detail & Related papers  (2025-10-18T11:01:39Z)
• Dr.LLM: Dynamic Layer Routing in LLMs [55.11953638340419]
  Dr.LLM is a retrofittable framework that equips pretrained models with lightweight per-layer routers deciding to skip, execute, or repeat a block.<n>On ARC (logic) and DART (math), Dr.LLM improves accuracy by up to +3.4%p while saving 5 layers per example on average.
  arXiv  Detail & Related papers  (2025-10-14T17:51:26Z)
• A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning [40.6234318894435]
  Large language models split into two families: reasoning-centric LLMs and agentic LLMs.<n>This divide arises from fundamentally different training objectives, leading to mismatched strengths and inefficiency on simple queries.<n>We present Adaptive Agent Foundation Model (A$2$FM), a unified framework that follows a route-then-align principle.
  arXiv  Detail & Related papers  (2025-10-13T17:08:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.