ALRA: Adaptive Local Relational Alignment for Logit-Based Pre-training Distillation of Autoregressive Language Models

• URL: http://arxiv.org/abs/2609.03355v1
• Date: Thu, 03 Sep 2026 04:26:39 GMT
• Title: ALRA: Adaptive Local Relational Alignment for Logit-Based Pre-training Distillation of Autoregressive Language Models
• Abstract summary: Local selection can miss tokens that the student considers likely, while student-only selection can rely on an inaccurate ranking early in training.<n>We propose Adaptive Local Alignment (ALRA), a position-specific framework combining student proposals with teacher guidance.<n>ALRA exceeds the strongest competing distillation baseline by 0.94 and 0.83 percentage points and improves over pre-training distillation by 2.31 and 2.91 points, respectively.
• Score: 4.5311655360445515
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Logit-based knowledge distillation for autoregressive language models usually aligns teacher and student next-token distributions over the entire vocabulary. However, this global objective overlooks relative preferences among likely token alternatives. Existing local approaches often select candidate tokens from either the teacher or the student alone. Teacher-only selection can miss tokens that the student considers likely, while student-only selection can rely on an inaccurate ranking early in training. We propose Adaptive Local Relational Alignment (ALRA), a position-specific framework combining student proposals with teacher guidance. At each valid prediction position, the student proposes likely tokens, while the teacher's most probable token is included as an anchor. ALRA adjusts the number of selected tokens according to how broadly the teacher distributes probability within this candidate set relative to the current batch. Adaptive Local Divergence retains the mass-matching term and separately matches the relative token distributions within the selected and remaining vocabulary regions. Unlike the exact full-vocabulary decomposition, it replaces the teacher-mass coefficients of the two conditional terms with unit coefficients, preventing either term from being downweighted solely because its region has low teacher probability. Student-Weighted Pairwise Relational Alignment emphasizes high-probability token pairs with small student probability gaps and gives less weight to unlikely or clearly separated pairs. Experiments on The Pile with randomly initialized 200M- and 500M-parameter students across nine zero-shot benchmarks yield average accuracies of 36.62% and 37.40%. ALRA exceeds the strongest competing distillation baseline by 0.94 and 0.83 percentage points and improves over pre-training without distillation by 2.31 and 2.91 points, respectively.

Related papers

• Mismatch Matters: On-Policy Distillation Beyond Token Agreement [27.396104103072844]
  We show that mismatch tokens can be mainly categorized into two types: student-excess tokens and student-deficit tokens.<n>We propose TIDE, which applies bounded Hellinger shaping to suppress the most severe sampled excesses and an analytic teacher top-$K$ injection to restore deficient probability mass.
  arXiv  Detail & Related papers  (2026-08-10T16:53:14Z)
• PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation [1.1440940588722308]
  On-policy self-distillation (OPSD) uses a privileged teacher to supervise a reasoning model on prefixes sampled from its own rollouts.<n>We introduce Privileged Adaptation from Student Trajectories (PAST)<n>PAST treats each completed student trajectory as additional privileged information for the OPSD teacher while leaving the student's distillation prefixes unchanged.
  arXiv  Detail & Related papers  (2026-08-09T14:20:12Z)
• Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models [23.382646724389545]
  On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions.<n>We propose emphFisher-Projected On-Policy Distillation (FP-OPD), which distills only locally realizable teacher corrections.
  arXiv  Detail & Related papers  (2026-08-02T14:16:14Z)
• Prediction-Only Distillation in Linear and Logistic Regression [61.07540493350384]
  Self-distillation is typically studied when the student is retrained on the teacher's original training inputs.<n>We study SD in this prediction-only regime through a fresh-X prediction-mixed scheme.<n>For binary logistic regression, we show that prediction mixing can outperform both the teacher and the pure-distilled.
  arXiv  Detail & Related papers  (2026-07-16T20:45:56Z)
• Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions [80.33276868889341]
  Z-Reward decouples reasoning-heavy judgment from efficient reward deployment.<n>We show that Z-Reward can serve as a differentiable reward signal for text-to-image optimization.
  arXiv  Detail & Related papers  (2026-06-08T06:20:12Z)
• X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation [19.064775527374618]
  Cross-tokenizer knowledge distillation allows a student model to learn from teachers with vocabularies.<n>Logit-based methods either use only the correct-token probability, missing the full 'dark knowledge' in the teacher's distribution.<n>We propose X-Token, an approach with two complementary loss formulations targeting these issues.
  arXiv  Detail & Related papers  (2026-05-20T19:59:31Z)
• When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning [45.79647925282674]
  On-policy self-distillation (OPSD) trains a student on its own rollouts using a privileged teacher.<n>Existing entropy-based OPD methods relax this uniformity by modulating token-level supervision with teacher entropy.<n>We propose Position-Weighted On-Policy Self-Distillation (PW-OPSD), which applies an increasing position weight while keeping the same student rollout, privileged teacher pass, and clipped forward-KL target as OPSD.
  arXiv  Detail & Related papers  (2026-05-20T18:14:03Z)
• Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation [49.117085054884676]
  On-policy distillation trains a student model on its own rollouts using dense feedback from a stronger teacher.<n>We operationalize this principle through a trajectory-specific release rule.<n> Experimental results across strong-to-weak distillation tasks indicate that this release rule consistently outperforms standard full-trajectory OPD.
  arXiv  Detail & Related papers  (2026-05-13T15:05:30Z)
• Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment [82.00769536768509]
  Rank-Surprisal Ratio is a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory.<n>We demonstrate its practical utility in both trajectory selection and teacher selection.
  arXiv  Detail & Related papers  (2026-01-20T18:58:10Z)
• Judging with Confidence: Calibrating Autoraters to Preference Distributions [56.17041629492863]
  We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population.<n>We present two learning methods tailored to different data conditions.<n>Our results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution.
  arXiv  Detail & Related papers  (2025-09-30T20:36:41Z)
• Selective Classification Can Magnify Disparities Across Groups [89.14499988774985]
  We find that while selective classification can improve average accuracies, it can simultaneously magnify existing accuracy disparities. Increasing abstentions can even decrease accuracies on some groups. We train distributionally-robust models that achieve similar full-coverage accuracies across groups and show that selective classification uniformly improves each group.
  arXiv  Detail & Related papers  (2020-10-27T08:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.