EviBack: Search-Agent Reinforcement Learning via Evidence-Constrained Teacher Backoff

• URL: http://arxiv.org/abs/2607.23955v2
• Date: Tue, 28 Jul 2026 06:31:52 GMT
• Title: EviBack: Search-Agent Reinforcement Learning via Evidence-Constrained Teacher Backoff
• Abstract summary: EviBack is an evidence- constrained Teacher backoff that supplies auxiliary super- vision to such groups.<n>A fully automated, end-to-end GPT- 5.5-assisted APE pipeline starts from a manually authored single-prompt dual-task Teacher.
• Score: 12.413369909547699
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning enables Agentic RAG systems to learn multi-turn search from verifiable outcome rewards, but all- zero rollout groups provide no comparative signal and may hide useful search behavior. We present EviBack, an evidence- constrained Teacher backoff that supplies auxiliary super- vision to such groups while preserving verifiable Actor re- wards. It separates evidence assessment from answer refine- ment, preventing reference answers from overriding evidence- insufficiency judgments. A fully automated, end-to-end GPT- 5.5-assisted APE pipeline starts from a manually authored single-prompt dual-task Teacher, automatically partitions and labels rollout data, and performs ablation, task decomposition, evaluation, and selection to produce a gated two-stage Teacher. Compared with the manual design, the resulting Teacher im- proves downstream F1 and valid-answer rate while reduc- ing search, duplicate queries, and forced termination. Across seven open-domain QA benchmarks and three Qwen3 scales, EviBack improves F1 over Search-R1 and raises both single- and multi-hop macro F1. We guarantee that the code will be made publicly available at a later stage.

Related papers

• RecurSE: Bounded Recursive Self-Evaluation for LLM Rubric Judges [48.831446371402414]
  RecurSE pairs a trainable judge with a synchronized policy-copy checker that audits the judge's reasoning against meta-rubrics to supply a scalar process reward.<n>We show that RecurSE achieves consistent generalization gains across held-out medical, pairwise, summarization, and professional benchmarks.
  arXiv  Detail & Related papers  (2026-08-25T08:35:04Z)
• SearchMaster: Grounded and Regulated Self-Play for Search Agents [57.537666863340284]
  Training search agents requires high-quality search data.<n> Existing pipelines often depend on human-written tasks, expert demonstrations, or stronger teacher models.<n>We present SearchMaster, a self-play framework that trains a single LLM from search tasks it generates, solves, and verifies in a local search environment.
  arXiv  Detail & Related papers  (2026-08-03T07:29:11Z)
• EviSD: Evidence-Conditioned Self-Distillation for Search-Augmented Agents [28.95182429824442]
  We propose EviSD, an evidence-conditioned self-distillation framework.<n>It uses instance-level supporting evidence as privileged information for search actions and golden answers.<n>EviSD achieves the highest macro-average Exact Match in all evaluated settings.
  arXiv  Detail & Related papers  (2026-08-02T16:22:27Z)
• DynaKRAG: A Unified Framework for Learnable Evidence Control in Multi-Hop Retrieval-Augmented Generation [12.710406983346013]
  DynaKRAG formulates multi-hop evidence acquisition as state-conditioned control over atomic evidence operations.<n>With Qwen2.5-7B-Instruct, DynaKRAG achieves F1 scores of 0.5998 on HotpotQA, 0.5340 on 2Wiki, and 0.3061 on MuSiQue.
  arXiv  Detail & Related papers  (2026-07-07T17:09:36Z)
• Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifiers [25.040339775169883]
  We propose a verifier-free training method in which the base policy, conditioned on the rubric, serves as the teacher for the unconditioned student.<n>Across Qwen-2.5 (3B, 7B) and Qwen3-Thinking (4B, 8B) models on medical and science domains, RGSD achieves satisfaction comparable to judge-based rubricO.
  arXiv  Detail & Related papers  (2026-06-10T17:53:19Z)
• HKVM-RAG: Key-Value-Separated Hypergraph Evidence Organization for Multi-Hop RAG [5.749820809225301]
  Multi-hop RAG poses a problem beyond passage matching: under fixed retrieval budgets, a system must organize retrieved text into evidence units that expose answer chains.<n>We present HKVM-RAG, a key-value-separated evidence-organization layer.
  arXiv  Detail & Related papers  (2026-06-05T12:31:13Z)
• QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards [68.15832368034815]
  We present a framework that co-designs queries and rubrics.<n>QUBRIC achieves a +5.5 point gain on ArenaHard over the SFT baseline.
  arXiv  Detail & Related papers  (2026-06-02T17:53:04Z)
• CoSearch: Joint Training of Reasoning and Document Ranking via Reinforcement Learning for Agentic Search [51.911048955965136]
  CoSearch is a framework that jointly trains a multi-step reasoning agent and a generative document ranking model.<n>Our results show that joint training of the reasoning agent and retrieval system is both feasible and strongly performant.
  arXiv  Detail & Related papers  (2026-04-19T17:48:17Z)
• The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearning [54.67958855362658]
  We propose a dynamic framework that stress tests unlearning using complex structured queries.<n>Our approach first elicits knowledge from the target model (pre-unlearning) and constructs targeted probes, ranging from simple queries to multi-hop chains.<n>Our framework enables practical and scalable evaluation of unlearning methods without the need for manual construction of forget test sets.
  arXiv  Detail & Related papers  (2026-03-11T19:51:33Z)
• Questions Are All You Need to Train a Dense Passage Retriever [123.13872383489172]
  ART is a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data. It uses a new document-retrieval autoencoding scheme, where (1) an input question is used to retrieve a set of evidence documents, and (2) the documents are then used to compute the probability of reconstructing the original question.
  arXiv  Detail & Related papers  (2022-06-21T18:16:31Z)
• Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering [40.58976291178477]
  We introduce a simple, fast, and unsupervised iterative evidence retrieval method. Despite its simplicity, our approach outperforms all the previous methods on the evidence selection task. When these evidence sentences are fed into a RoBERTa answer classification component, we achieve state-of-the-art QA performance.
  arXiv  Detail & Related papers  (2020-05-04T00:19:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.