PathRouter: Aligning Rewards with Retrieval Quality in Agentic Graph Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2606.16409v1
• Date: Mon, 15 Jun 2026 08:48:04 GMT
• Title: PathRouter: Aligning Rewards with Retrieval Quality in Agentic Graph Retrieval-Augmented Generation
• Abstract summary: Agentic GraphRAG trains language-model agents to iteratively retrieve and reason over graph-structured evidence.<n>We present Path, a path-aware training framework for agentic GraphRAG.
• Score: 41.730631759393525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic GraphRAG trains language-model agents to iteratively retrieve and reason over graph-structured evidence, enabling more accurate and context-aware decision-making by efficiently navigating complex information networks. However, outcome-only reinforcement learning suffers from \textit{\textbf{answer-path reward aliasing}}, where correct answers may come from shortcuts rather than useful evidence paths. It also exhibits \textit{\textbf{search-update ambiguity}}, as scalar trajectory-level feedback does not indicate which retrieval actions to adjust. To mitigate these shortcomings, we present PathRouter, a path-aware training framework for agentic GraphRAG. PathRouter jointly evaluates each trajectory along answer correctness and evidence-path overlap, yielding four trajectory categories with differentiated GRPO advantage scaling that suppresses shortcut reinforcement while preserving evidence-seeking behavior. For evidence-poor trajectories, a frozen gold-evidence teacher provides token-level KL guidance on reasoning and search-query tokens, excluding answer tokens to avoid direct response imitation. Experiments on six QA benchmarks across three model sizes show that PathRouter consistently improves answer F1 and evidence-path overlap, achieving average F1 gains of 3.1 on 3B and 4.9 on 7B models compared to a strong baseline.

Related papers

• GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning [27.83191719981676]
  We present textscGTA-RAG, a graph-trajectory-augmented RL framework for retrieval-augmented reasoning.<n>Method consistently outperforms RL-based RAG baselines with both Qwen2.5-3B and Qwen2.5-7B backbones.
  arXiv  Detail & Related papers  (2026-08-23T16:05:20Z)
• ExTra: Exploratory Trajectory Optimization for Language Model Reinforcement Learning [75.26064451955199]
  ExTra is a framework that extracts exploration signals from the model's own rollouts.<n>It improves Qwen3-1.7B over GRPO by about +5 points on pass@1 and +7 points on pass@16.
  arXiv  Detail & Related papers  (2026-06-23T15:51:39Z)
• TraceGraph: Shared Decision Landscapes for Diagnosing and Improving Agent Trajectories [61.51984029109709]
  We introduce TraceGraph, a graph-based framework that turns released multi-model agent trajectories into shared decision landscapes.<n>For each task, TraceGraph builds a graph over observable action-observation states from pooled rollouts before model identity is introduced.<n>It then overlays outcome-informed productive cores and trap regions, and summarizes each rollout with three events: Access, Trap exposure, and Repair.
  arXiv  Detail & Related papers  (2026-05-29T13:40:31Z)
• STAR: Semantic-Tuned and Tail-Adaptive Retriever for Graph-Augmented Generation [69.66045538889777]
  We propose STAR, a semantic-tuned and tail-adaptive retriever for GraphRAG.<n> STAR consistently outperforms baselines, achieving average retrieval performance gains of 1.8%.
  arXiv  Detail & Related papers  (2026-04-11T10:16:51Z)
• GraphWalker: Agentic Knowledge Graph Question Answering via Synthetic Trajectory Curriculum [22.56991897386462]
  Agentic knowledge graph question answering (KGQA) requires an agent to iteratively interact with knowledge graphs (KGs)<n>Existing approaches often restrict agent exploration, while current training pipelines usually confine reasoning to predefined trajectories.<n>This paper proposes textitGraphWalker,' a novel agentic KGQA framework that addresses these challenges through textitAutomated Trajectory Synthesis and textitStage-wise Fine-tuning
  arXiv  Detail & Related papers  (2026-03-30T14:56:59Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• HELP: HyperNode Expansion and Logical Path-Guided Evidence Localization for Accurate and Efficient GraphRAG [53.30561659838455]
  Large Language Models (LLMs) often struggle with inherent knowledge boundaries and hallucinations.<n>Retrieval-Augmented Generation (RAG) frequently overlooks structural interdependencies essential for multi-hop reasoning.<n>Help achieves competitive performance across multiple simple and multi-hop QA benchmarks and up to a 28.8$times$ speedup over leading Graph-based RAG baselines.
  arXiv  Detail & Related papers  (2026-02-24T14:05:29Z)
• Breaking the Static Graph: Context-Aware Traversal for Robust Retrieval-Augmented Generation [12.71443292660797]
  We propose CatRAG, Context-Aware Traversal for robust RAG.<n>CatRAG builds on the HippoRAG 2 architecture and transforms the static KG into a query-adaptive navigation structure.<n> Experiments across four multi-hop benchmarks demonstrate that CatRAG consistently outperforms state of the art baselines.
  arXiv  Detail & Related papers  (2026-02-02T11:13:38Z)
• Relink: Constructing Query-Driven Evidence Graph On-the-Fly for GraphRAG [10.60635000782353]
  We argue for a textitreason-and-construct paradigm and propose Relink, a framework that dynamically builds a query-specific evidence graph.<n>Relink achieves significant average improvements of 5.4% in EM and 5.2% in F1 over leading GraphRAG baselines.
  arXiv  Detail & Related papers  (2026-01-12T04:35:23Z)
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• PropRAG: Guiding Retrieval with Beam Search over Proposition Paths [15.346744525284604]
  PropRAG is a novel RAG framework that shifts from triples to context-rich propositions.<n>PropRAG achieves Recall@5 and F1 scores on 2Wiki, HotpotQA, and MuSiQue.
  arXiv  Detail & Related papers  (2025-04-25T04:47:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.