Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflicts

• URL: http://arxiv.org/abs/2606.30518v1
• Date: Mon, 29 Jun 2026 16:25:13 GMT
• Title: Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflicts
• Abstract summary: We propose RAPS-DA, a regime-aware peer specialization framework.<n>At the sample level, conflicts are divided into three regimes, including Grounding, Arbitration, and Resistance.<n>At the token level, a dual-layer selector uses inter-teacher disagreement, student-teacher divergence, and student entropy to filter uninformative or unstable tokens.
• Score: 46.1469246256932
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-augmented generation (RAG) improves language models by grounding generation in external context. However, it can be fragile when the retrieved context conflicts with the model's parametric knowledge. Such conflicts span a reliability spectrum, ranging from reliable and partially reliable evidence to adversarial context. Existing remedies often handle such heterogeneous conflicts with regime-agnostic supervision, which can conflate incompatible learning signals across reliability regimes. To disentangle these signals, we propose RAPS-DA, a regime-aware peer specialization framework that addresses conflict at two complementary granularities. At the sample level, conflicts are divided into three regimes, including Grounding, Arbitration, and Resistance, with one same-scale peer specialist trained per regime from a shared base model. Each sample is then hard-routed to its regime-matched peer for on-policy reverse-KL supervision. At the token level, a dual-layer selector uses inter-teacher disagreement, student-teacher divergence, and student entropy to filter uninformative or unstable tokens, upweight confidently misaligned ones, and gradually focus supervision on high-conflict tokens as the student matures. Gains stem from specialization at a fixed model scale, not from a stronger teacher, and the peer specialists exist only during training, so the deployed student requires no regime labels or peer access. Experiments on five conflict scenarios and two out-of-distribution benchmarks show RAPS-DA surpasses all prompting, decoding, fine-tuning, RL, and single-teacher baselines.

Related papers

• Purified OPSD: On-Policy Self-Distillation Without Losing How to Think [52.879387744920415]
  We find that on-policy self-distillation (OPSD) consistently fails on long chain-of-thought (long-CoT) reasoning models.<n>We propose a two-step solution: first, we construct a reference-only teacher to isolate the non-transferable component of the supervision signal.<n>Second, we use pointwise mutual information (PMI) as the mechanism to transform this residual into a well-formed PMI target distribution.
  arXiv  Detail & Related papers  (2026-07-02T14:33:07Z)
• To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition [87.22988227382329]
  Multimodal emotion recognition (MER) benefits from combining text, audio, and vision, yet standard fusion often fails when modalities conflict.<n>We propose Dual-Path Conflict Resolution (DCR), a unified framework that learns when to fuse and when to drop modalities.
  arXiv  Detail & Related papers  (2026-05-06T13:11:33Z)
• Diagnosing Knowledge Conflict in Multimodal Long-Chain Reasoning [78.86309644343295]
  Multimodal large language models (MLLMs) in long chain-of-thought reasoning often fail when different knowledge sources provide conflicting signals.<n>We formalize these failures under a unified notion of knowledge conflict, distinguishing input-level objective conflict from process-level effective conflict.<n>Our findings provide a mechanism-level view of multimodal reasoning under knowledge conflict and enable principled diagnosis and control of long-CoT failures.
  arXiv  Detail & Related papers  (2026-02-16T07:10:44Z)
• "The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework [16.96094045628127]
  Chain-of-Thought (CoT) reasoning empowers Large Language Models (LLMs) with remarkable capabilities but typically requires prohibitive parameter scales.<n>CoT distillation has emerged as a promising paradigm to transfer reasoning prowess into compact Student Models (SLMs)<n>We introduce COMPACT, a framework that adaptively fuses supervisions from different teachers by dynamically weighting teacher gradients.
  arXiv  Detail & Related papers  (2026-01-20T14:05:19Z)
• When in Doubt, Deliberate: Confidence-Based Routing to Expert Debate for Sexism Detection [7.299050989302629]
  We propose a framework to address the combined effects of (i) underrepresentation, (ii) noise, and (iii) conceptual ambiguity in both data and model predictions.<n>Our approach achieves state-of-the-art results across several benchmarks, with a +2.72% improvement in F1 on Tasks EXIST 2025 Task 1.1, and a gains of +4.48% and +1.30% on EDOS A and B, respectively.
  arXiv  Detail & Related papers  (2025-12-21T05:48:57Z)
• Targeting Misalignment: A Conflict-Aware Framework for Reward-Model-based LLM Alignment [5.900494456937422]
  Reward-model-based fine-tuning is a central paradigm in aligning Large Language Models with human preferences.<n>This paper investigates a novel framework to identify and mitigate such misalignment by treating the fine-tuning process as a form of knowledge integration.
  arXiv  Detail & Related papers  (2025-12-10T00:52:21Z)
• Debiased Dual-Invariant Defense for Adversarially Robust Person Re-Identification [52.63017280231648]
  Person re-identification (ReID) is a fundamental task in many real-world applications such as pedestrian trajectory tracking.<n>Person ReID models are highly susceptible to adversarial attacks, where imperceptible perturbations to pedestrian images can cause entirely incorrect predictions.<n>We propose a dual-invariant defense framework composed of two main phases.
  arXiv  Detail & Related papers  (2025-11-13T03:56:40Z)
• GuardFed: A Trustworthy Federated Learning Framework Against Dual-Facet Attacks [56.983319121358555]
  Federated learning (FL) enables privacy-preserving collaborative model training but remains vulnerable to adversarial behaviors.<n>We introduce the Dual-Facet Attack (DFA), a novel threat model that concurrently undermines predictive accuracy and group fairness.<n>We propose GuardFed, a self-adaptive defense framework that maintains a fairness-aware reference model using a small amount of clean server data.
  arXiv  Detail & Related papers  (2025-11-12T13:02:45Z)
• TRiCo: Triadic Game-Theoretic Co-Training for Robust Semi-Supervised Learning [15.638836465479619]
  TRiCo is a novel triadic game-theoretic co-training framework that rethinks the structure of semi-supervised learning.<n>By addressing key limitations in existing SSL frameworks, TRiCo provides a principled and generalizable solution.
  arXiv  Detail & Related papers  (2025-09-25T20:10:41Z)
• Provable Unrestricted Adversarial Training without Compromise with Generalizability [44.02361569894942]
  Adversarial training (AT) is widely considered as the most promising strategy to defend against adversarial attacks. The existing AT methods often achieve adversarial robustness at the expense of standard generalizability. We propose a novel AT approach called Provable Unrestricted Adversarial Training (PUAT)
  arXiv  Detail & Related papers  (2023-01-22T07:45:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.