Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation

• URL: http://arxiv.org/abs/2609.01091v2
• Date: Wed, 02 Sep 2026 12:09:21 GMT
• Title: Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation
• Abstract summary: A teacher biased by a system prompt can generate semantically clean training data, such as numeric sequences, that still causes a downstream student to inherit the hidden preference.<n>We propose and validate trait-direction drift as a mechanism for subliminal learning.<n>We propose probe-space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction during distillation.
• Score: 6.96679232120357
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Beyond intended capabilities, model distillation can transfer hidden traits from a teacher. A teacher biased by a system prompt can generate semantically clean training data, such as numeric sequences, that still causes a downstream student to inherit the hidden preference, a phenomenon known as subliminal learning. Prior work has identified several parts of this process. How the signal builds up during training and produces behavioral transfer remains unclear, making targeted mitigation difficult. We propose and validate trait-direction drift as a mechanism for subliminal learning: biased generation creates measurable preference gaps in teacher data, and student-recognizable gaps induce trait-aligned updates during supervised fine-tuning that accumulate into behavioral transfer. Guided by this mechanism, we propose probe-space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction during distillation. The method substantially reduces hidden-trait transfer, preserving task performance: for example, it lowers malicious-response transfer from 29.55% to 6.45% with low main-task accuracy cost, and consistently suppresses animal-preference transfer across the main Qwen setting. The preference-gap, training-trajectory, and intervention evidence links subliminal learning to trait-direction drift and motivates corridor regularization as a targeted control during distillation.

Related papers

• Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall [79.59596652272923]
  We propose Switch Distillation, a simple mid-training objective that distills on tokens where the teacher is confident.<n>Relative to standard NTP, it achieves 1.61-1.71x the reasoning performance and 1.13-1.19x the knowledge and commonsense performance.
  arXiv  Detail & Related papers  (2026-09-01T17:00:30Z)
• Subliminal Steering: Stronger Encoding of Hidden Signals [5.13724383217928]
  Subliminal learning describes a student language model inheriting a behavioral bias by fine-tuning on seemingly innocuous data.<n>We introduce subliminal steering, a variant of subliminal learning in which the teacher's bias is implemented not via a system prompt, but through a steering vector trained to maximize the likelihood of a set of target samples.<n>We show that subliminal steering transfers complex multi-word biases, whereas prior work focused on single-word preferences, demonstrating a large scope of subliminally transferrable signals.
  arXiv  Detail & Related papers  (2026-04-28T15:51:55Z)
• Sustained Gradient Alignment Mediates Subliminal Learning in a Multi-Step Setting: Evidence from MNIST Auxiliary Logit Distillation Experiment [1.0026496861838448]
  In the MNIST auxiliary logit distillation experiment, a student can acquire an unintended teacher trait despite distilling only on no-class logits.<n>Under a single-step gradient descent assumption, subliminal learning theory attributes this effect to alignment between the trait and distillation gradients.<n>We show that gradient alignment remains weakly but consistently positive throughout training and causally contributes to trait acquisition.
  arXiv  Detail & Related papers  (2026-04-28T15:46:18Z)
• Distillation Traps and Guards: A Calibration Knob for LLM Distillability [54.90137955363471]
  We propose a post-hoc calibration method that enables control over a teacher's distillability via reinforcement fine-tuning.<n>Our objective combines task utility, KL anchor, and across-tokenizer calibration reward.<n> Experiments show that students distilled from distillable teachers outperform SFT and KD baselines.
  arXiv  Detail & Related papers  (2026-04-21T01:22:35Z)
• Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation [0.7788319765644828]
  We show that unsafe agent behaviors can transfer subliminally through model distillation across two experimental settings.<n>In our primary setting, we construct a teacher agent exhibiting a strong deletion bias, and distill it into a student using only trajectories from ostensibly safe tasks.<n>We replicate the threat model in a native Bash environment, replacing API tool calls with shell commands and operationalizing the bias as a preference for issuing chmod as the first permission-related command.
  arXiv  Detail & Related papers  (2026-04-16T22:23:01Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• FlowSteer: Guiding Few-Step Image Synthesis with Authentic Trajectories [82.90132015584359]
  ReFlow has theoretical consistency with flow matching but suboptimal performance in practical scenarios.<n>We propose FlowSteer, a method unlocks the potential of ReFlow-based distillation by guiding the student along teacher's authentic generation trajectories.
  arXiv  Detail & Related papers  (2025-11-24T07:13:23Z)
• Classifier-Free Guidance inside the Attraction Basin May Cause Memorization [16.752023123940674]
  Diffusion models are prone to exactly reproduce images from the training data.<n>Memorized images are high in image quality and well-aligned with the conditioning mechanism.
  arXiv  Detail & Related papers  (2024-11-23T15:36:03Z)
• On the Surprising Effectiveness of Attention Transfer for Vision Transformers [118.83572030360843]
  Conventional wisdom suggests that pre-training Vision Transformers (ViT) improves downstream performance by learning useful representations. We investigate this question and find that the features and representations learned during pre-training are not essential.
  arXiv  Detail & Related papers  (2024-11-14T18:59:40Z)
• Multi-Granularity Semantic Revision for Large Language Model Distillation [66.03746866578274]
  We propose a multi-granularity semantic revision method for LLM distillation. At the sequence level, we propose a sequence correction and re-generation strategy. At the token level, we design a distribution adaptive clipping Kullback-Leibler loss as the distillation objective function. At the span level, we leverage the span priors of a sequence to compute the probability correlations within spans, and constrain the teacher and student's probability correlations to be consistent.
  arXiv  Detail & Related papers  (2024-07-14T03:51:49Z)
• Contrastive Supervised Distillation for Continual Representation Learning [18.864301420659217]
  A neural network model is sequentially learned to alleviate catastrophic forgetting in visual search tasks. Our method, called Contrastive Supervised Distillation (CSD), reduces feature forgetting while learning discriminative features.
  arXiv  Detail & Related papers  (2022-05-11T13:20:47Z)
• Self-supervised Transformer for Deepfake Detection [112.81127845409002]
  Deepfake techniques in real-world scenarios require stronger generalization abilities of face forgery detectors. Inspired by transfer learning, neural networks pre-trained on other large-scale face-related tasks may provide useful features for deepfake detection. In this paper, we propose a self-supervised transformer based audio-visual contrastive learning method.
  arXiv  Detail & Related papers  (2022-03-02T17:44:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.