Distilling Reasoning Without Knowledge: A Framework for Reliable LLMs

• URL: http://arxiv.org/abs/2603.14458v1
• Date: Sun, 15 Mar 2026 16:06:54 GMT
• Title: Distilling Reasoning Without Knowledge: A Framework for Reliable LLMs
• Abstract summary: We propose a modular framework that separates planning from factual retrieval and answer synthesis.<n>A lightweight student planner is trained via a teacher-student framework to generate structured decompositions.<n>We evaluate the proposed framework on SEAL-0, an extremely challenging benchmark for search-augmented LLMs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fact-seeking question answering with large language models (LLMs) remains unreliable when answers depend on up-to-date or conflicting information. Although retrieval-augmented and tool-using LLMs reduce hallucinations, they often rely on implicit planning, leading to inefficient tool usage. We propose a modular framework that explicitly separates planning from factual retrieval and answer synthesis. A lightweight student planner is trained via a teacher-student framework to generate structured decompositions consisting of abstract reasoning steps and searchable fact requests. The supervision signals contain only planning traces and fact requests, without providing factual answers or retrieved evidence. At inference, the planner produces plans, while prompt-engineered modules perform retrieval and response synthesis. We evaluate the proposed framework on SEAL-0, an extremely challenging benchmark for search-augmented LLMs. Results show that supervised planning improves both accuracy and latency compared to monolithic reasoning models and prompt-based tool-augmented frameworks, demonstrating that explicitly learned planning structures are essential for reliable fact-seeking LLMs.

Related papers

• Planner-Centric Reinforcement Learning for Deep Research with Structure-Aware Reward [4.833783769369]
  DecomposeR is a planner-centric deep research framework that represents research plans as typed directed acyclic graphs (DAGs)<n>We train a Qwen3-8B model in two stages: planner reinforcement learning (RL) first learns graph structure and query decomposition to improve research planning, and answerer reinforcement learning (RL) then learns branch-level execution and final synthesis conditioned on the learned plan.<n> Experiments show that DecomposeR improves over strong comparable open baselines by 5.1-8.0 points on popular long-form benchmarks due to improved planning and answering capabilities.
  arXiv  Detail & Related papers  (2026-05-29T04:18:55Z)
• PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models [52.48858778580074]
  Planning is a fundamental capability for large language models (LLMs)<n>PlanningBench is a framework for generating scalable, diverse, verifiable planning data for both evaluation and training.
  arXiv  Detail & Related papers  (2026-05-20T08:10:15Z)
• STRIDE: Strategic Iterative Decision-Making for Retrieval-Augmented Multi-Hop Question Answering [9.69666629029613]
  Multi-hop question answering (MHQA) enables accurate answers to complex queries by retrieving and reasoning over evidence dispersed across multiple documents.<n>Existing MHQA approaches mainly rely on iterative retrieval-augmented generation, which suffer from the following two major issues.<n>We propose STRIDE, a framework that separates strategic planning, dynamic control, and grounded execution.
  arXiv  Detail & Related papers  (2026-04-19T12:19:43Z)
• DUPLEX: Agentic Dual-System Planning via LLM-Driven Information Extraction [11.920443665332629]
  Large Language Models (LLMs) provide semantic flexibility for robotic task planning.<n>Their susceptibility to hallucination and logical inconsistency limits their reliability in long-horizon domains.<n>We propose a dual-system neuro-symbolic architecture that strictly confines the LLM to schema-guided information extraction.
  arXiv  Detail & Related papers  (2026-03-25T03:57:44Z)
• Plan Then Retrieve: Reinforcement Learning-Guided Complex Reasoning over Knowledge Graphs [52.16166558205338]
  Graph-RFT is a novel two-stage reinforcement fine-tuning KGQA framework with a 'plan-KGsearch-and-Websearch-during-think' paradigm.<n>It enables LLMs to perform autonomous planning and adaptive retrieval scheduling across KG and web sources under incomplete knowledge conditions.
  arXiv  Detail & Related papers  (2025-10-23T16:04:13Z)
• Self-Reflective Planning with Knowledge Graphs: Enhancing LLM Reasoning Reliability for Question Answering [9.601307470705732]
  We propose Self-Reflective Planning (SRP), a framework that synergizes large language models with knowledge graphs.<n>In the planning process, SRP first searches for references to guide planning and reflection.<n>After retrieving knowledge from KGs through a reasoning path, it implements iterative reflection by judging the retrieval result and editing the reasoning path until the answer is correctly retrieved.
  arXiv  Detail & Related papers  (2025-05-26T01:59:00Z)
• DataPuzzle: Breaking Free from the Hallucinated Promise of LLMs in Data Analysis [10.98270220152657]
  Large language models (LLMs) are increasingly applied to multi-modal data analysis.<n>The prevailing Prompt-to-Answer'' paradigm treats LLMs as black-box analysts.<n>We propose DataPuzzle, a conceptual multi-agent framework that decomposes complex questions.
  arXiv  Detail & Related papers  (2025-04-14T09:38:23Z)
• Language Agents Meet Causality -- Bridging LLMs and Causal World Models [50.79984529172807]
  We propose a framework that integrates causal representation learning with large language models. This framework learns a causal world model, with causal variables linked to natural language expressions. We evaluate the framework on causal inference and planning tasks across temporal scales and environmental complexities.
  arXiv  Detail & Related papers  (2024-10-25T18:36:37Z)
• Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs [59.76268575344119]
  We introduce a novel framework for enhancing large language models' (LLMs) planning capabilities by using planning data derived from knowledge graphs (KGs) LLMs fine-tuned with KG data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval.
  arXiv  Detail & Related papers  (2024-06-20T13:07:38Z)
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
  arXiv  Detail & Related papers  (2024-06-19T00:28:58Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [68.29746557968107]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans.<n> Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.