Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents

• URL: http://arxiv.org/abs/2605.23590v1
• Date: Fri, 22 May 2026 12:59:16 GMT
• Title: Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agents
• Abstract summary: We introduce Co-ReAct, a rubric-guided action-selection framework.<n>At each decision step, Co-ReAct injects a rubric into the agent's context to guide the next Reason-or-Act decision.<n>We train a dedicated rubric generator with GRPO to make this guidance reliable.
• Score: 6.89645931986174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ReAct-style agents for search-intensive, multi-step reasoning tasks rely largely on their own internal judgment to decide what evidence to seek, which reasoning or action step to take next, and when to stop, often producing shallow, redundant, or poorly targeted trajectories. Prior work has explored rubrics as external quality signals, but existing uses are mostly evaluative rather than action-guiding: rubrics typically serve as training-time rewards or post-hoc evaluators of completed outputs, and in deep-research settings they are often coarse-grained and report-level rather than step-level. We introduce Co-ReAct, a rubric-guided action-selection framework that uses rubrics as step-level guidance during inference. At each decision step, Co-ReAct injects a rubric into the agent's context to guide the next Reason-or-Act decision, specifying what the agent should target in evidence seeking, search, reasoning, or self-evaluation. To make this guidance reliable, we train a dedicated rubric generator with GRPO. Unlike prior pairwise or binary preference formulations, our objective optimizes a list-wise Spearman rank-correlation reward against multi-judge expert consensus rankings, encouraging rubrics that are discriminative rather than merely plausible. On DeepResearchBench and SQA-CS-V2, Co-ReAct consistently improves over ReAct and representative test-time compute baselines across search agents built on both 8B/14B open-source and frontier closed-source base models. The trained rubric generator can also serve as a drop-in component that improves these baselines without changing their underlying decision mechanisms. Our code is publicly available at https://github.com/ZBWpro/Co-ReAct.

Related papers

• Shared Prefixes, Better Credit: Adaptive Routing for Multi-Agent Reasoning [72.49938949082117]
  Multi-agent reasoning (MAR) improves reasoning reliability through iterative solution exchange and refinement.<n>Existing adaptive MAR methods typically learn routing decisions from query-level labels or trajectory-level returns.<n>We propose TreeCredit, a shared-operator credit assignment framework for efficient adaptive MAR.
  arXiv  Detail & Related papers  (2026-08-03T14:22:23Z)
• RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents [37.33108522642245]
  We propose RICE-PO, a critic-free policy optimization framework that converts retrieval interactions into localized learning signals.<n>On BRIGHT and BEIR, RICE-PO consistently outperforms prompt-based agents and group-based RL baselines under the same retriever setting.
  arXiv  Detail & Related papers  (2026-05-25T21:56:29Z)
• RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards [76.17893114021757]
  Deep research systems that plan, search, evaluate evidence, and synthesize long-form reports lack ground-truth answers and span many tool-augmented decisions.<n>In this work, we argue that rubrics should serve not merely as final-answer evaluators, but as the shared interface that structures policy execution, judge feedback, and agent memory.<n>We introduceEM, a rubric-guided reinforcement learning framework that combines stagewise policy decomposition with reflection-based meta-policy evolution.
  arXiv  Detail & Related papers  (2026-05-11T17:40:38Z)
• SRR-Judge: Step-Level Rating and Refinement for Enhancing Search-Integrated Reasoning in Search Agents [30.92763154920672]
  We introduce SRR-Judge, a framework for reliable step-level assessment of reasoning and search actions.<n>SRR-Judge provides fine-grained guidance for search-integrated reasoning and enables efficient post-training annotation.<n> Empirically, SRR-Judge delivers more reliable step-level evaluations than much larger models such as DeepSeek-V3.1.
  arXiv  Detail & Related papers  (2026-02-08T02:07:41Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation [80.12435680651488]
  We propose a pipeline to train human-preference-aligned query-specific rubric generators tailored for DeepResearch report generation.<n>We first construct a dataset of DeepResearch-style annotated queries with human preferences over paired reports, and train rubric generators via reinforcement learning.<n>We empirically show that our proposed rubric generators deliver more discriminative and better human-aligned supervision than existing rubric design strategies.
  arXiv  Detail & Related papers  (2026-02-03T15:09:56Z)
• Thinking Forward and Backward: Multi-Objective Reinforcement Learning for Retrieval-Augmented Reasoning [137.33138614095435]
  Retrieval-augmented generation (RAG) has proven to be effective in mitigating hallucinations in large language models.<n>Recent efforts have incorporated search-based interactions into RAG, enabling iterative reasoning with real-time retrieval.<n>We propose Bi-RAR, a novel retrieval-augmented reasoning framework that evaluates each intermediate step jointly in both forward and backward directions.
  arXiv  Detail & Related papers  (2025-11-12T08:29:39Z)
• SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework [7.37561751991963]
  We propose a process-supervised multi-agent framework to bridge the gap between retriever and generator.<n>The proposed framework is modular and plug-and-play, requiring no modification to the retriever or generator.
  arXiv  Detail & Related papers  (2025-09-17T09:09:28Z)
• ReAct: Temporal Action Detection with Relational Queries [84.76646044604055]
  This work aims at advancing temporal action detection (TAD) using an encoder-decoder framework with action queries. We first propose a relational attention mechanism in the decoder, which guides the attention among queries based on their relations. Lastly, we propose to predict the localization quality of each action query at inference in order to distinguish high-quality queries.
  arXiv  Detail & Related papers  (2022-07-14T17:46:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.