KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search

• URL: http://arxiv.org/abs/2606.29863v1
• Date: Mon, 29 Jun 2026 06:56:18 GMT
• Title: KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search
• Abstract summary: We propose KbSD (Knowledge boundary Self-Distillation), a framework that tackles the limitation through dense token-level supervision.<n>KbSD constructs a hint-augmented teacher, architecturally identical to the student, that receives explicit knowledge boundary signals.<n>Experiments show that KbSD consistently improves both task accuracy and hallucination mitigation over strong baselines.
• Score: 13.024432404905198
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic search equips large language models with dynamic retrieval abilities, but existing reinforcement learning methods remain limited by reward sparsity in knowledge boundary calibration -- deciding when to trust parametric memory, when to rely on retrieved evidence, and when to abstain. Binary rewards can penalize undesirable outcomes, but provide little guidance on the reasoning process required to make calibrated decisions across different knowledge states. To address this, we propose KbSD (Knowledge boundary Self-Distillation), a framework that tackles this limitation through dense token-level supervision, outcome-level sparse rewards, and quadrant-adaptive optimization. KbSD constructs a hint-augmented teacher, architecturally identical to the student, that receives explicit knowledge boundary signals -- including parametric certainty, retrieval quality, and ground-truth answers -- to generate calibrated reasoning demonstrations. This information-asymmetric self-distillation enables dense supervision without requiring a larger external model. To further account for the heterogeneous reasoning distributions across knowledge states, we introduce a quadrant-adaptive distillation objective: reverse KL for concentrated integration, forward KL for diverse refusal, and Pareto-optimal bidirectional KL for asymmetric quadrants requiring both precision and coverage. Experiments on multiple benchmarks show that KbSD consistently improves both task accuracy and hallucination mitigation over strong baselines, with the largest gains appearing in the challenging quadrants where sparse rewards are least informative.

Related papers

• ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning [74.19730275669227]
  ARISE-RL is a novel full-cycle self-evolution framework for training open-ended agents.<n>We present Reward-Gated Self-Evolution Distillation (RG-SED), which distills a memory-augmented variant of the same policy back into itself.<n>We also present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning.
  arXiv  Detail & Related papers  (2026-09-01T10:54:13Z)
• DIAG: Diagnostic Iterative Alignment and Generation for Data-Efficient Mathematical Preference Distillation [7.6963470169750465]
  We propose a Diagnostic Iterative Alignment and Generation framework to increase informative supervision and focus training near the student's current competence boundary.<n>Experiments show that DIAG boosts yield across iterations and delivers stronger reasoning performance under an iso-effective training budget.
  arXiv  Detail & Related papers  (2026-08-24T05:09:12Z)
• Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation [60.55792673956761]
  We propose textbfRubric-Conditioned Self-Distillation, a framework that incorporates rubrics as structured, fine-grained feedback for on-policy self-distillation.<n>We evaluate on a diverse suite of science reasoning benchmarks and results show that rubric-conditioned self-distillation effectively converts rubric-level criteria into token-level guidance.
  arXiv  Detail & Related papers  (2026-06-17T17:54:04Z)
• Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning [63.24624171679711]
  Large language model (LLM)-based agents often make suboptimal tool-use decisions.<n>We propose TRUST, which incorporates uncertainty quantification into reward design as a repulsive force for maintaining uncertainty separation.
  arXiv  Detail & Related papers  (2026-06-05T07:08:34Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• KARL: Mitigating Hallucinations in LLMs via Knowledge-Boundary-Aware Reinforcement Learning [46.007379982024474]
  KARL is a novel framework that aligns an LLM's abstention behavior with its evolving knowledge boundary.<n>It achieves a superior accuracy-hallucination trade-off, effectively suppressing hallucinations while maintaining high accuracy across both in-distribution and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-04-03T11:30:56Z)
• Optimizing In-Context Demonstrations for LLM-based Automated Grading [31.353360036776976]
  GUIDE (Grading Using Iteratively Designed Exemplars) is a framework that reframes exemplar selection and refinement as a boundary-focused optimization problem.<n>We show that GUIDE significantly outperforms standard retrieval baselines in experiments in physics, chemistry, and pedagogical content knowledge.
  arXiv  Detail & Related papers  (2026-02-28T04:52:38Z)
• Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoning [31.629261193485053]
  Large reasoning models (LRMs) have emerged as a powerful paradigm for solving complex real-world tasks.<n>Most existing outcome-only RLVR pipelines rely almost exclusively on a binary correctness signal and largely ignore the model's intrinsic uncertainty.<n>We propose EGPO, a metacognitive entropy calibration framework that explicitly integrates intrinsic uncertainty into RLVR for enhancing LRMs.
  arXiv  Detail & Related papers  (2026-02-26T08:40:06Z)
• Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaboration [49.9937230730202]
  We propose Search-R2, a novel Actor-Refiner collaboration framework that enhances reasoning through targeted intervention.<n>Our approach decomposes the generation process into an Actor, which produces initial reasoning trajectories.<n>We show that Search-R2 consistently outperforms strong RAG and RL-based baselines across model scales.
  arXiv  Detail & Related papers  (2026-02-03T15:32:09Z)
• CATNIP: LLM Unlearning via Calibrated and Tokenized Negative Preference Alignment [14.853204323785334]
  Existing approaches, rooted in Gradient Ascent (GA), often degrade general domain knowledge while relying on retention data or curated contrastive pairs.<n>We develop a principled method that rescales unlearning effects in proportion to the model's token-level confidence.<n>Our work enables effective unlearning without requiring retention data or contrastive unlearning response pairs.
  arXiv  Detail & Related papers  (2026-02-02T21:23:54Z)
• ASK: Adaptive Self-improving Knowledge Framework for Audio Text Retrieval [19.94287753279928]
  The dominant paradigm for Audio-Text Retrieval (ATR) relies on mini-batch-based contrastive learning.<n>The Gradient Locality Bottleneck (GLB) structurally prevents models from leveraging out-of-batch knowledge.<n>The Representation-Drift Mismatch (RDM) is where a static knowledge base becomes progressively misaligned with the evolving model, turning guidance into noise.
  arXiv  Detail & Related papers  (2025-12-11T14:48:30Z)
• Sycophancy Mitigation Through Reinforcement Learning with Uncertainty-Aware Adaptive Reasoning Trajectories [58.988535279557546]
  We introduce textbf sycophancy Mitigation through Adaptive Reasoning Trajectories.<n>We show that SMART significantly reduces sycophantic behavior while preserving strong performance on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2025-09-20T17:09:14Z)
• Divide-Then-Align: Honest Alignment based on the Knowledge Boundary of RAG [51.120170062795566]
  We propose Divide-Then-Align (DTA) to endow RAG systems with the ability to respond with "I don't know" when the query is out of the knowledge boundary.<n>DTA balances accuracy with appropriate abstention, enhancing the reliability and trustworthiness of retrieval-augmented systems.
  arXiv  Detail & Related papers  (2025-05-27T08:21:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.