CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning

• URL: http://arxiv.org/abs/2608.03673v1
• Date: Tue, 04 Aug 2026 13:49:12 GMT
• Title: CausalOPD: First-Wrong-Step Supervision for Distilling Causal Chain Reasoning
• Abstract summary: CausalOPD is a curriculum online process distillation framework.<n>A knowledge-augmented teacher first provides trajectories grounded in domain-specific causal rules.<n>A causal-stage curriculum advances from evidence-level to mechanism-level and conclusion-level errors.
• Score: 5.241146164724621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many critical reasoning tasks, including clinical diagnosis, legal judgment, and industrial fault diagnosis, require step-dependent causal chains in which early errors propagate and correct conclusions can mask invalid reasoning. Although large language models perform well on such tasks, privacy, latency, and controllability motivate distillation into locally deployable models. Standard trajectory imitation does not correct process errors on the student's own rollout distribution. We propose CausalOPD, a curriculum online process distillation framework. A knowledge-augmented teacher first provides trajectories grounded in domain-specific causal rules, entity relations, and structural constraints. The student then generates on-policy trajectories, and the teacher identifies the first wrong step, defined as the earliest transition that verifiably violates available constraints. Starting from the verified prefix, short-horizon reinforcement learning repairs this localized failure. A causal-stage curriculum advances from evidence-level to mechanism-level and conclusion-level errors, following their propagation order. Across three domains, CausalOPD improves average path correctness by 23.4 percentage points over sequence-level online process distillation and reduces the right-label-wrong-reasoning rate from 15.7% to 4.4%. The domain-specific 8B students also surpass both evaluated proprietary references in path correctness across all domains.

Related papers

• BOUND: Brief-Guided Corrective Preference Distillation at Search-Control Boundaries [82.41764922522565]
  BOUND is a brief-guided corrective preference distillation framework for persistent search drift.<n>It outperforms Trajectory SFT by 5.6 EM points on Bamboogle and 4.8 accuracy points on BrowseComp-Plus.
  arXiv  Detail & Related papers  (2026-08-09T15:32:16Z)
• DiagLoop: A Counterfactual Data Flywheel with Stage-Localized Reinforcement for Diagnostic LLMs [5.241146164724621]
  We present DiagLoop, a counterfactual data flywheel that converts codified physical relations or clinical guidelines into training supervision.<n>A training-only teacher proposes counterfactual worlds by varying causes, contexts, and observations, while an independent hybrid checker admits only valid worlds.<n>For nonterminal failures, a bounded repair probes downstream competence, and the resulting weakness profile guides subsequent data generation.
  arXiv  Detail & Related papers  (2026-08-04T13:49:41Z)
• Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming [5.951101028540166]
  Chain-of-thought (CoT) prompting enables large language models to tackle multi-step reasoning tasks.<n>We present Reason Popper-ly, a neurosymbolic framework that uses inductive logic programming (ILP) to learn relation composition rules.
  arXiv  Detail & Related papers  (2026-07-25T03:24:05Z)
• H$^2$SD: Hybrid Hindsight Self-Distillation [61.71131241473028]
  Reinforcement learning with verifiable rewards (RLVR) provides reliable outcome supervision for language model reasoning.<n>Existing self-distillation methods add a privileged teacher but typically assign it a fixed role.<n>We introduce Hybrid Hindsight Self-Distillation ($mathrmH2mathrmSD$), which jointly adapts teacher context and update strategy to trajectory correctness.
  arXiv  Detail & Related papers  (2026-07-21T10:47:27Z)
• Purified OPSD: On-Policy Self-Distillation Without Losing How to Think [52.879387744920415]
  We find that on-policy self-distillation (OPSD) consistently fails on long chain-of-thought (long-CoT) reasoning models.<n>We propose a two-step solution: first, we construct a reference-only teacher to isolate the non-transferable component of the supervision signal.<n>Second, we use pointwise mutual information (PMI) as the mechanism to transform this residual into a well-formed PMI target distribution.
  arXiv  Detail & Related papers  (2026-07-02T14:33:07Z)
• Trajectory-Refined Distillation [25.346810464266497]
  Trajectory-Refined Distillation (TRD) is a trajectory-level correction method that revises the student's rollout under the teacher guidance while within on-policy support.<n> TRD can also be applied to on-policy self-distillation (OPSD), a parameter-sharing variant that uses the student model conditioned on privileged informations as the teacher.
  arXiv  Detail & Related papers  (2026-06-07T03:17:25Z)
• When Attribution Patching Lies: Diagnosis and a Second-Order Correction [9.338965648455238]
  We show that the dominant error stems from the non-linearities in the downstream network rather than local curvature at the patched component.<n>This insight yields three practical tools: (i) a reliability score to detect untrustworthy estimates, (ii) error bounds quantifying potential attribution mis-specifications, and (iii) a Hessian-vector-product correction that eliminates the leading-order error with only one additional backward pass.
  arXiv  Detail & Related papers  (2026-06-05T05:20:32Z)
• Validity-Calibrated Reasoning Distillation [4.277471273303775]
  Reasoning distillation aims to transfer multi-step reasoning capabilities from large language models to smaller, more efficient ones.<n>Recent methods rely on static teacher-student hierarchies and frame distillation as trajectory imitation.<n>We propose validity-calibrated reasoning distillation, a framework that treats reasoning distillation as a problem of local learning-signal allocation.
  arXiv  Detail & Related papers  (2026-04-14T12:32:12Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering [71.15346406323827]
  We introduce PRIME, a benchmark for evaluating verifiers on Process-Outcome Alignment verification.<n>We find that current verifiers frequently fail to detect derivation flaws.<n>We propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME.
  arXiv  Detail & Related papers  (2026-02-12T04:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.