SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation

• URL: http://arxiv.org/abs/2609.12579v1
• Date: Fri, 11 Sep 2026 08:33:52 GMT
• Title: SCOPE-OPSD: Fisher-Conditioned Privileged Subspaces for On-Policy Self-Distillation
• Abstract summary: SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor.<n>It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules.<n>Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD.
• Score: 22.56867206175605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: On-policy self-distillation (OPSD) scores student-generated prefixes with a solution-conditioned self-teacher, yet transfers supervision only through next-token probabilities. We ask whether the aligned final-layer discrepancy offers a useful second channel, and how to test that channel without confusing its geometry with auxiliary strength. SCOPE-OPSD projects the privileged teacher-student residual onto a frozen rank-64 factor estimated from residual covariance and language-model-head Fisher sensitivity. It reuses the forwards already required by OPSD and adds neither rollouts nor inference-time modules. A matched Random control preserves the structured factor's rank and nonzero spectrum and uses per-arm gradient-RMS calibration, isolating the effect of the data-dependent orientation. Across the complete 25/50/75/100-step trajectories for Qwen3-1.7B, 4B, and 8B, Structured is never below Pure OPSD, with strict gains in 11 of the 12 model-checkpoint combinations and an exact tie at 4B step 25. Structured also exceeds matched Random in 10 of the 12 combinations. At step 75 on Qwen3-1.7B, Structured exceeds matched Random by 1.39 Macro Avg@12 points in each of two independent training reruns. A cross-fitted diagnostic also shows 4.40 times greater held-out privileged-gap capture than the matched random orientation. The results support a compact, Fisher-conditioned privileged subspace for short-budget OPSD.

Related papers

• A Unified Per-Token Gating Family for On-Policy Distillation: FKL/RKL Mixing with Multi-Channel and Bias Coefficients [6.033644084569457]
  Per-token gating of forward/reverse KL losses has become a standard technique for on-policy knowledge distillation (OPD)<n> EOPD and ToDi each fix a single gating signal and a single gating direction, and the two have never been compared directly.<n>We introduce a four-coefficient parameterization_t = sigma(a * h_t + b * u(x) + c + d * gap_t) in which direction-aligned proxies of EOPD and ToDi appear as one-dimensional (1D) restrictions.
  arXiv  Detail & Related papers  (2026-09-10T16:22:35Z)
• On-Policy Self-Distillation in Diffusion Models [89.32656931795293]
  Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint rewards do not specify how an intermediate denoising prediction should change.<n>We introduce DiffusionOPSD as an on-policy self-distillation framework that converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries.<n>Across SD 3.5-M and the step-distilled Z-Image-Turbo, our approach achieves the best final held-out scores in 19 of 20 reward-matched settings across two backbones and ten evaluators.
  arXiv  Detail & Related papers  (2026-08-25T14:55:24Z)
• Distill Skills into Weights, Not Prompts: Abstract Skills as Privileged Signals for On-Policy Self-Distillation [7.296659715681798]
  Reinforcement learning with verifiable rewards yields no group-relative signal when rollout groups are uniformly correct or uniformly wrong.<n>We propose SKALD (Skill-Anchored Latent Distillation), an on-policy self-distillation framework that uses two context views of the same Qwen3-Base model.
  arXiv  Detail & Related papers  (2026-08-10T16:43:18Z)
• WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training [48.290951447286744]
  WDL-OPD is a mixture-constrained co-training method with two trainable policies.<n>An anchor policy generates every rollout, an auxiliary policy evaluates the same visited states, and a geometric mixture of their token distributions is matched to a frozen teacher by reverse KL.<n>We show that freezing the auxiliary recovers an anchor-plus-contrast proxy target closely related to OPD$2$ and W2S-OPD, whereas joint training creates branch-level degrees of freedom that a static delta cannot express.
  arXiv  Detail & Related papers  (2026-08-10T11:22:53Z)
• BOUND: Brief-Guided Corrective Preference Distillation at Search-Control Boundaries [82.41764922522565]
  BOUND is a brief-guided corrective preference distillation framework for persistent search drift.<n>It outperforms Trajectory SFT by 5.6 EM points on Bamboogle and 4.8 accuracy points on BrowseComp-Plus.
  arXiv  Detail & Related papers  (2026-08-09T15:32:16Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning [55.264863369127774]
  Current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.<n>We propose Hidden-Align, an auxiliary loss function that aligns the last-layer hidden states of correct rollouts at the anchor token during RL training, with zero overhead in both training and inference.<n>On eight mathematical reasoning benchmarks, Hidden-Align improves average pass@1 over the DAPO baseline by 3.8, 6.2, and 5.4 percentage points on Qwen3-1.7B, 4B, and 14B respectively, with consistent pass@k gains across all three scales.
  arXiv  Detail & Related papers  (2026-06-02T06:51:15Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schrödinger Samplers [68.9946633641494]
  Flow matching and Schrdinger bridges define probability, yet their inference grids are usually or inherited from one-endpoint diffusion.<n>We derive a conditional-marginal entropy-rate objective for bridge-aware discretization, separating endpoint-conditioned bridge geometry from marginal flow evolution.<n>On EDM/CIFAR-10, the entropic time-discretization gives the best tested five-step FID (186.3 pm 4.0 versus 200.5 pm 2.9 for linear and 238.0 pm 5.3 for cosine)
  arXiv  Detail & Related papers  (2026-05-15T16:11:10Z)
• Adaptive Consensus in LLM Ensembles via Sequential Evidence Accumulation: Automatic Budget Identification and Calibrated Commit Signals [0.0]
  DASE produces a commit-type routing partition that generalises across benchmarks.<n> Adaptive stopping, not injection bandwidth, drives accuracy.<n>Injection-based methods exhibit an inverted-U accuracy-vs-inference trajectory.
  arXiv  Detail & Related papers  (2026-05-05T19:24:10Z)
• The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models [0.0]
  Empirically, three independent leaderboards (Open LLM v2, an extended 12-benchmark suite, LiveBench) all have d_eff in [2.86, 4.80] on their competitive frontier.<n>A submodular greedy algorithm with the Nemhauser (1 - 1/e) guarantee finds a stable core of 4 benchmarks.<n>A counterfactual validation across 12 internal benchmarks and 27 categories confirms that the eigenstructure predicts which evaluations are irreplaceable.
  arXiv  Detail & Related papers  (2026-04-15T08:56:58Z)
• Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes [3.305265383862785]
  We present an empirical study of Low-Rank Adaptation (LoRA) in sequential transformer encoder finetuning with companion representation probes.<n>In five full-validation BERT-base reruns on an RTE->MRPC->CoLA->SST-2 sequence, full fine-tuning yields 19.9% +/- average forgetting, whereas standard LoRA (r, query/value modules) yields 0.6% +/-1.4%.
  arXiv  Detail & Related papers  (2026-03-29T14:14:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.