One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement

• URL: http://arxiv.org/abs/2604.25444v1
• Date: Tue, 28 Apr 2026 09:52:21 GMT
• Title: One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinement
• Abstract summary: Large Language Models (LLMs) often fail to utilize their latent reasoning capabilities.<n>We propose ReQueR, a modular framework that treats reasoning elicitation as an inference-time alignment task.
• Score: 37.27419953581617
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) often fail to utilize their latent reasoning capabilities due to a distributional mismatch between ambiguous human inquiries and the structured logic required for machine activation. Existing alignment methods either incur prohibitive $O(N)$ costs by fine-tuning each model individually or rely on static prompts that fail to resolve query-level structural complexity. In this paper, we propose ReQueR (\textbf{Re}inforcement \textbf{Que}ry \textbf{R}efinement), a modular framework that treats reasoning elicitation as an inference-time alignment task. We train a specialized Refiner policy via Reinforcement Learning to rewrite raw queries into explicit logical decompositions, treating frozen LLMs as the environment. Rooted in the classical Zone of Proximal Development from educational psychology, we introduce the Adaptive Solver Hierarchy, a curriculum mechanism that stabilizes training by dynamically aligning environmental difficulty with the Refiner's evolving competence. ReQueR yields consistent absolute gains of 1.7\%--7.2\% across diverse architectures and benchmarks, outperforming strong baselines by 2.1\% on average. Crucially, it provides a promising paradigm for one-to-many inference-time reasoning elicitation, enabling a single Refiner trained on a small set of models to effectively unlock reasoning in diverse unseen models. Code is available at https://github.com/newera-xiao/ReQueR.

Related papers

• STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answering [9.69666629029613]
  Multi-hop question answering (MHQA) enables accurate answers to complex queries by retrieving and reasoning over evidence dispersed across multiple documents.<n>Existing MHQA approaches mainly rely on iterative retrieval-augmented generation, which suffer from the following two major issues.<n>We propose STRIDE, a framework that separates strategic planning, dynamic control, and grounded execution.
  arXiv  Detail & Related papers  (2026-04-19T12:19:43Z)
• PLUME: Latent Reasoning Based Universal Multimodal Embedding [52.35354073629127]
  Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model.<n>Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings.<n>We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states.
  arXiv  Detail & Related papers  (2026-04-02T14:04:53Z)
• Reinforcing Structured Chain-of-Thought for Video Understanding [49.3086326382764]
  We introduce Summary-Driven Reinforcement Learning (SDRL)<n>SDRL is a single-stage RL framework that obviates the need forSupervised Fine-Tuning (SFT)<n>Our method achieves state-of-the-art performance on seven public VideoQA datasets.
  arXiv  Detail & Related papers  (2026-03-26T22:11:14Z)
• AgenticRS-EnsNAS: Ensemble-Decoupled Self-Evolving Architecture Search [10.111487060179252]
  This work introduces Ensemble-Decoupled Architecture Search.<n>It predicts system-level performance from single-learner evaluation.<n>It reduces per-candidate search cost from O(M) to O(1) while maintaining O(M) deployment cost only for validated winners.
  arXiv  Detail & Related papers  (2026-03-20T14:57:15Z)
• When should I search more: Adaptive Complex Query Optimization with Reinforcement Learning [26.489185170468062]
  We propose a novel RL framework called Adaptive Complex Query Optimization (ACQO)<n>Our framework is designed to adaptively determine when and how to expand the search process.<n>ACQO achieves state-of-the-art performance on three complex query benchmarks, significantly outperforming established baselines.
  arXiv  Detail & Related papers  (2026-01-29T03:16:53Z)
• R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learning [32.16683059021539]
  Large reasoning models (LRMs) aim to solve diverse and complex problems through structured reasoning.<n>Recent advances in group-based policy optimization methods have shown promise in enabling stable advantage estimation without reliance on process-level annotations.<n>We propose a reinforcement learning mechanism named emphtextbfR3 that along three directions: (1) a emphcross-context underlinetextbfReplay strategy that maintains the intra-group advantage, (2) an emphin-context self-underlinetextbfReflection mechanism
  arXiv  Detail & Related papers  (2026-01-27T13:55:34Z)
• MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval [50.30107119622642]
  Large Language Models (LLMs) excel at reasoning and generation but are inherently limited by static pretraining data.<n>Retrieval-Augmented Generation (RAG) addresses this issue by grounding LLMs in external knowledge.<n>MarAG-R1 is a reinforcement-learned multi-tool RAG framework that enables LLMs to dynamically coordinate multiple retrieval mechanisms.
  arXiv  Detail & Related papers  (2025-10-31T15:51:39Z)
• Step-Aware Policy Optimization for Reasoning in Diffusion Large Language Models [57.42778606399764]
  Diffusion language models (dLLMs) offer a promising, non-autoregressive paradigm for text generation.<n>Current reinforcement learning approaches often rely on sparse, outcome-based rewards.<n>We argue that this stems from a fundamental mismatch with the natural structure of reasoning.
  arXiv  Detail & Related papers  (2025-10-02T00:34:15Z)
• GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning [53.894789613838654]
  We introduce SEED-Bench-R1, a benchmark with complex real-world videos requiring balanced perception and reasoning.<n>Using SEED-Bench-R1, we find that standard GRPO, while improving answer accuracy, often reduces logical coherence between reasoning steps and answers, with only a 57.9% consistency rate.<n>We propose GRPO-CARE, a consistency-aware RL framework optimizing both answer correctness and reasoning coherence without explicit supervision.
  arXiv  Detail & Related papers  (2025-06-19T08:49:13Z)
• Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Router [9.580226379350737]
  Multi-step reasoning has proven essential for enhancing the problem-solving capabilities of Large Language Models.<n>Yet, many reasoning steps are relatively simple and can be handled by more efficient smaller-scale language models.<n>We propose R2-Reasoner, a novel framework that enables collaborative reasoning across heterogeneous LLMs.
  arXiv  Detail & Related papers  (2025-06-06T09:18:56Z)
• Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition [95.54406667705999]
  Pangu Embedded is an efficient Large Language Model (LLM) reasoner developed on Ascend Neural Processing Units (NPUs)<n>It addresses the significant computational costs and inference latency challenges prevalent in existing reasoning-optimized LLMs.<n>It delivers rapid responses and state-of-the-art reasoning quality within a single, unified model architecture.
  arXiv  Detail & Related papers  (2025-05-28T14:03:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.