Knowledgeable-r1: Policy Optimization for Knowledge Exploration in Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2506.05154v1
• Date: Thu, 05 Jun 2025 15:34:15 GMT
• Title: Knowledgeable-r1: Policy Optimization for Knowledge Exploration in Retrieval-Augmented Generation
• Authors: Chenyu Lin, Yilin Wen, Du Su, Fei Sun, Muhan Chen, Chenfu Bao, Zhonghou Lv,
• Abstract summary: Retrieval-augmented generation (RAG) is a mainstream method for improving performance on knowledge-intensive tasks.<n>We propose Knowledgeable-r1, that uses joint sampling and define multi policy distributions in knowledge capability exploration.<n> Experiments show that Knowledgeable-r1 significantly enhances robustness and reasoning accuracy in both parameters and contextual conflict tasks.
• Score: 6.870247946243668
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-augmented generation (RAG) is a mainstream method for improving performance on knowledge-intensive tasks. However,current RAG systems often place too much emphasis on retrieved contexts. This can lead to reliance on inaccurate sources and overlook the model's inherent knowledge, especially when dealing with misleading or excessive information. To resolve this imbalance, we propose Knowledgeable-r1 that using joint sampling and define multi policy distributions in knowledge capability exploration to stimulate large language models'self-integrated utilization of parametric and contextual knowledge. Experiments show that Knowledgeable-r1 significantly enhances robustness and reasoning accuracy in both parameters and contextual conflict tasks and general RAG tasks, especially outperforming baselines by 17.07% in counterfactual scenarios and demonstrating consistent gains across RAG tasks. Our code are available at https://github.com/lcy80366872/ knowledgeable-r1.

Related papers

• DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router [57.28685457991806]
  DeepSieve is an agentic RAG framework that incorporates information sieving via LLM-as-a-knowledge-router.<n>Our design emphasizes modularity, transparency, and adaptability, leveraging recent advances in agentic system design.
  arXiv  Detail & Related papers  (2025-07-29T17:55:23Z)
• RAG in the Wild: On the (In)effectiveness of LLMs with Mixture-of-Knowledge Retrieval Augmentation [45.679455112940175]
  Retrieval-augmented generation (RAG) enhances large language models (LLMs) by integrating external knowledge retrieved at inference time.<n>We evaluated RAG systems using MassiveDS, a large-scale datastore with mixture of knowledge, and identified critical limitations.
  arXiv  Detail & Related papers  (2025-07-26T20:57:24Z)
• Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs [69.10441885629787]
  Retrieval-Augmented Generation (RAG) lifts the factuality of Large Language Models (LLMs) by injecting external knowledge.<n>It falls short on problems that demand multi-step inference; conversely, purely reasoning-oriented approaches often hallucinate or mis-ground facts.<n>This survey synthesizes both strands under a unified reasoning-retrieval perspective.
  arXiv  Detail & Related papers  (2025-07-13T03:29:41Z)
• CoRe-MMRAG: Cross-Source Knowledge Reconciliation for Multimodal RAG [53.950029990391066]
  Cross-source knowledge textbfReconciliation for Multimodal RAG (CoRe-MMRAG)<n>We propose a novel end-to-end framework that effectively reconciles inconsistencies across knowledge sources.<n>Experiments on KB-VQA benchmarks show that CoRe-MMRAG achieves substantial improvements over baseline methods.
  arXiv  Detail & Related papers  (2025-06-03T07:32:40Z)
• KARE-RAG: Knowledge-Aware Refinement and Enhancement for RAG [63.82127103851471]
  Retrieval-Augmented Generation (RAG) enables large language models to access broader knowledge sources.<n>We demonstrate that enhancing generative models' capacity to process noisy content is equally critical for robust performance.<n>We present KARE-RAG, which improves knowledge utilization through three key innovations.
  arXiv  Detail & Related papers  (2025-06-03T06:31:17Z)
• Learning to Route Queries Across Knowledge Bases for Step-wise Retrieval-Augmented Reasoning [60.84901522792042]
  Multimodal Retrieval-Augmented Generation (MRAG) has shown promise in mitigating hallucinations in Multimodal Large Language Models (MLLMs)<n>We propose R1, a novel MRAG framework that learns to decide when and where to retrieve knowledge based on the evolving reasoning state.<n>R1- can adaptively and effectively leverage diverse KBs, reducing unnecessary retrievals and improving both efficiency and accuracy.
  arXiv  Detail & Related papers  (2025-05-28T08:17:57Z)
• Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generation [32.30660197797758]
  We introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD)<n>We demonstrate superior accuracy and significant computational efficiency improvements compared to the previous surrogate-based method.<n>Our mechanistic analysis reveals specific attention heads and multilayer perceptron (MLP) layers responsible for context attribution.
  arXiv  Detail & Related papers  (2025-05-22T09:04:03Z)
• Know3-RAG: A Knowledge-aware RAG Framework with Adaptive Retrieval, Generation, and Filtering [9.814926166669366]
  We propose Know3-RAG, a knowledge-aware RAG framework that leverages structured knowledge from knowledge graphs (KGs) to guide three core stages of the RAG process, including retrieval, generation, and filtering.<n> Experiments on multiple open-domain QA benchmarks demonstrate that Know3-RAG consistently outperforms strong baselines, significantly reducing hallucinations and enhancing answer reliability.
  arXiv  Detail & Related papers  (2025-05-19T03:25:18Z)
• Self-Routing RAG: Binding Selective Retrieval with Knowledge Verbalization [97.72503890388866]
  We propose Self-Routing RAG (SR-RAG), a novel framework that binds selective retrieval with knowledge verbalization.<n>SR-RAG enables an LLM to dynamically decide between external retrieval and verbalizing its own parametric knowledge.<n>We introduce dynamic knowledge source inference via nearest neighbor search to improve the accuracy of knowledge source decision.
  arXiv  Detail & Related papers  (2025-04-01T17:59:30Z)
• Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models [20.605487145370752]
  We find that imperfect retrieval augmentation is inevitable, common, and harmful.<n>We propose Astute RAG, a novel RAG approach designed to be resilient to imperfect retrieval augmentation.<n>Experiments with Gemini and Claude demonstrate the superior performance of Astute RAG compared to previous robustness-enhanced RAG approaches.
  arXiv  Detail & Related papers  (2024-10-09T17:59:58Z)
• Multi-Source Knowledge Pruning for Retrieval-Augmented Generation: A Benchmark and Empirical Study [46.55831783809377]
  Retrieval-augmented generation (RAG) is increasingly recognized as an effective approach to mitigating the hallucination of large language models (LLMs)<n>We develop PruningRAG, a plug-and-play RAG framework that uses multi-granularity pruning strategies to more effectively incorporate relevant context and mitigate the negative impact of misleading information.
  arXiv  Detail & Related papers  (2024-09-03T03:31:37Z)
• Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation [14.448198170932226]
  Think-on-Graph 2.0 (ToG-2) is a hybrid RAG framework that iteratively retrieves information from both unstructured and structured knowledge sources.<n>ToG-2 alternates between graph retrieval and context retrieval to search for in-depth clues relevant to the question.<n>It achieves overall state-of-the-art (SOTA) performance on 6 out of 7 knowledge-intensive datasets with GPT-3.5.
  arXiv  Detail & Related papers  (2024-07-15T15:20:40Z)
• KAT: A Knowledge Augmented Transformer for Vision-and-Language [56.716531169609915]
  We propose a novel model - Knowledge Augmented Transformer (KAT) - which achieves a strong state-of-the-art result on the open-domain multimodal task of OK-VQA. Our approach integrates implicit and explicit knowledge in an end to end encoder-decoder architecture, while still jointly reasoning over both knowledge sources during answer generation. An additional benefit of explicit knowledge integration is seen in improved interpretability of model predictions in our analysis.
  arXiv  Detail & Related papers  (2021-12-16T04:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.