Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverable

• URL: http://arxiv.org/abs/2608.26958v1
• Date: Thu, 27 Aug 2026 10:59:03 GMT
• Title: Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverable
• Abstract summary: We show a second effect: larger datasets can make subtle teacher-specific signals easier to detect in the trained student.<n>In a controlled setup inspired by subliminal learning, a teacher induced to express a target trait generates restricted off-task data.<n>Students trained on different amounts of independent off-task data are evaluated in a separate domain.
• Score: 10.484473401339464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling model-generated data is usually viewed as improving distillation: more examples should increase coverage, reduce noise, and produce stronger students. We show a second effect: larger datasets can make subtle teacher-specific signals easier to detect in the trained student, even when examples are off-task and never mention the trait. In a controlled setup inspired by subliminal learning, a teacher induced to express a target trait generates restricted off-task data, such as number-only completions. Students trained on different amounts of independent off-task data are evaluated in a separate domain, with matched no-trait controls isolating target-specific transfer. Our main finding is that larger independent datasets make the teacher's induced trait stand out more clearly in the student's later behavior. Other plausible traits may also strengthen with scale, but the target usually grows more. When the small-scale student already favors the target, scaling mainly amplifies that behavior; when it favors a related or salient alternative, more data can shift behavior toward the intended trait. Analyses of learned LoRA updates show a parallel trend. These effects appear across model families, trait types, multi-trait settings, and cross-model transfer. Our results suggest that scaling generated distillation data should be paired with trait-aware curation and evaluation, even when the data appears off-task or benign.

Related papers

• Subliminal Learning is Non-Semantic Distillation [1.2891210250935148]
  Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.<n>It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data.<n>This presents challenges in ensuring AI systems remain predictable and are trained safely.
  arXiv  Detail & Related papers  (2026-08-06T08:18:13Z)
• Extracting alignment data in open models [50.81383232591576]
  We show that it is possible to extract significant amounts of alignment training data from a post-trained model.<n>This data is useful to steer the model to improve certain capabilities such as long-context reasoning, safety, instruction following, and maths.<n>We find that models readily regurgitate training data that was used in post-training phases such as SFT or RL.
  arXiv  Detail & Related papers  (2025-10-21T12:06:00Z)
• Subliminal Learning: Language models transmit behavioral traits via hidden signals in data [7.092398764321311]
  We study subliminal learning, a phenomenon where language models transmit behavioral traits via semantically unrelated data.<n>We prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions.<n>We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development.
  arXiv  Detail & Related papers  (2025-07-20T03:51:13Z)
• Self-Training and Multi-Task Learning for Limited Data: Evaluation Study on Object Detection [4.9914667450658925]
  Experimental results show the improvement of performance when using a weak teacher with unseen data for training a multi-task student. Despite the limited setup we believe the experimental results show the potential of multi-task knowledge distillation and self-training.
  arXiv  Detail & Related papers  (2023-09-12T14:50:14Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling [69.60713300418467]
  Learning to jump is a general recipe for generative modeling of various types of data. We demonstrate when learning to jump is expected to perform comparably to learning to denoise, and when it is expected to perform better.
  arXiv  Detail & Related papers  (2023-05-28T05:38:28Z)
• Distilling Robustness into Natural Language Inference Models with Domain-Targeted Augmentation [12.512147282842175]
  We investigate two complementary methods for improving the robustness of the resulting student models on out-of-distribution domains. The first approach augments the distillation with generated unlabelled examples that match the target distribution. The second method upsamples data points among the training set that are similar to the target distribution.
  arXiv  Detail & Related papers  (2023-05-22T14:37:05Z)
• Counterfactual Generation Under Confounding [24.503075567519048]
  A machine learning model, under the influence of observed or unobserved confounders in the training data, can learn spurious correlations. We propose a counterfactual generation method that learns to modify the value of any attribute in an image and generate new images given a set of observed attributes. Our method is computationally efficient, simple to implement, and works well for any number of generative factors and confounding variables.
  arXiv  Detail & Related papers  (2022-10-22T06:39:22Z)
• Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation [66.25738680429463]
  Knowledge Distillation (KD) for object detection aims to train a compact detector by transferring knowledge from a teacher model. We propose inconsistent knowledge distillation (IKD) which aims to distill knowledge inherent in the teacher model's counter-intuitive perceptions. Our method outperforms state-of-the-art KD baselines on one-stage, two-stage and anchor-free object detectors.
  arXiv  Detail & Related papers  (2022-09-20T16:36:28Z)
• Reminding the Incremental Language Model via Data-Free Self-Distillation [26.960750314663294]
  Incremental language learning with pseudo-data can alleviate catastrophic forgetting in neural networks. We propose reminding incremental language model via data-free self-distillation (DFSD) Our DFSD can exceed the previous state-of-the-art methods even if the maximum decrease in pseudo-data is 90%.
  arXiv  Detail & Related papers  (2021-10-17T07:27:43Z)
• Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition [65.77962788209103]
  We propose class subset learning by dividing the long-tailed data into multiple class subsets according to prior knowledge. It enforces the model to focus on learning the subset-specific knowledge. The proposed framework proved to be effective for the long-tailed retinal diseases recognition task.
  arXiv  Detail & Related papers  (2021-04-22T13:39:33Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.