Subliminal Learning: Language models transmit behavioral traits via hidden signals in data

• URL: http://arxiv.org/abs/2507.14805v1
• Date: Sun, 20 Jul 2025 03:51:13 GMT
• Title: Subliminal Learning: Language models transmit behavioral traits via hidden signals in data
• Abstract summary: We study subliminal learning, a phenomenon where language models transmit behavioral traits via semantically unrelated data.<n>We prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions.<n>We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development.
• Score: 7.092398764321311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study subliminal learning, a surprising phenomenon where language models transmit behavioral traits via semantically unrelated data. In our main experiments, a "teacher" model with some trait T (such as liking owls or being misaligned) generates a dataset consisting solely of number sequences. Remarkably, a "student" model trained on this dataset learns T. This occurs even when the data is filtered to remove references to T. We observe the same effect when training on code or reasoning traces generated by the same teacher model. However, we do not observe the effect when the teacher and student have different base models. To help explain our findings, we prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions, and demonstrate subliminal learning in a simple MLP classifier. We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development. Distillation could propagate unintended traits, even when developers try to prevent this via data filtering.

Related papers

• Verbalizing Subliminal Learning Effects Using Text Optimization [48.21170843630861]
  Subliminal learning is a phenomenon in which a distillation dataset transmits traits from the teacher model that are not legibly encoded in the dataset itself.<n>In this work, we use text optimization to detect subliminal learning effects and describe them as legible prompts.
  arXiv  Detail & Related papers  (2026-09-15T10:03:31Z)
• Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverable [10.484473401339464]
  We show a second effect: larger datasets can make subtle teacher-specific signals easier to detect in the trained student.<n>In a controlled setup inspired by subliminal learning, a teacher induced to express a target trait generates restricted off-task data.<n>Students trained on different amounts of independent off-task data are evaluated in a separate domain.
  arXiv  Detail & Related papers  (2026-08-27T10:59:03Z)
• Subliminal Learning is Non-Semantic Distillation [1.2891210250935148]
  Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.<n>It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data.<n>This presents challenges in ensuring AI systems remain predictable and are trained safely.
  arXiv  Detail & Related papers  (2026-08-06T08:18:13Z)
• Subliminal Learning is a LoRA Artifact [7.7398054555399085]
  Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data.<n>We show that subliminal learning is a LoRA artifact. When subliminal learning occurs, transmission has an inverted U-shaped relationship with LoRA rank; it also disappears with full finetuning.
  arXiv  Detail & Related papers  (2026-05-30T18:05:35Z)
• You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases [1.2369994923959178]
  Subliminal learning refers to the transmission of traits from a teacher to a student model via training on data unrelated to those traits.<n>We investigate whether transmission occurs through natural language paraphrases with fixed semantic content.<n>We find that training on paraphrases from a teacher system-prompted to love a particular animal increases a student's preference for that animal by up to 19 percentage points.
  arXiv  Detail & Related papers  (2026-03-10T11:21:14Z)
• Semi-Supervised Learning under General Causal Models [51.90307793476367]
  Semi-supervised learning (SSL) aims to train a machine learning model using both labelled and unlabelled data.<n>We propose a framework that works with general causal models in which the variables have flexible causal relations.<n>The learned causal generative model can generate synthetic labelled data for training a more accurate predictive model.
  arXiv  Detail & Related papers  (2025-10-26T07:46:38Z)
• Emergence of Linear Truth Encodings in Language Models [64.86571541830598]
  Large language models exhibit linear subspaces that separate true from false statements, yet the mechanism behind their emergence is unclear.<n>We introduce a transparent, one-layer transformer toy model that reproduces such truth subspaces end-to-end.<n>We study one simple setting in which truth encoding can emerge, encouraging the model to learn this distinction in order to lower the LM loss on future tokens.
  arXiv  Detail & Related papers  (2025-10-17T16:30:07Z)
• I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data? [76.15163242945813]
  Large language models (LLMs) have led many to conclude that they exhibit a form of intelligence.<n>We introduce a novel generative model that generates tokens on the basis of human-interpretable concepts represented as latent discrete variables.
  arXiv  Detail & Related papers  (2025-03-12T01:21:17Z)
• Understanding Self-Supervised Learning via Gaussian Mixture Models [19.51336063093898]
  We analyze self-supervised learning in a natural context: dimensionality reduction in Gaussian Mixture Models.<n>We show that vanilla contrastive learning is able to find the optimal lower-dimensional subspace even when the Gaussians are not isotropic.<n>In this setting we show that contrastive learning learns the subset of fisher-optimal subspace, effectively filtering out all the noise from the learnt representations.
  arXiv  Detail & Related papers  (2024-11-05T21:43:05Z)
• UnLearning from Experience to Avoid Spurious Correlations [3.283369870504872]
  We propose a new approach that addresses the issue of spurious correlations: UnLearning from Experience (ULE) Our method is based on using two classification models trained in parallel: student and teacher models. We show that our method is effective on the Waterbirds, CelebA, Spawrious and UrbanCars datasets.
  arXiv  Detail & Related papers  (2024-09-04T15:06:44Z)
• An iterated learning model of language change that mixes supervised and unsupervised learning [0.0]
  The iterated learning model is an agent model which simulates the transmission of of language from generation to generation.<n>In each iteration, a language tutor exposes a na"ive pupil to a limited training set of utterances, each pairing a random meaning with the signal that conveys it.<n>The transmission bottleneck ensures that tutors must generalize beyond the training set that they experienced.
  arXiv  Detail & Related papers  (2024-05-31T14:14:01Z)
• Stubborn Lexical Bias in Data and Models [50.79738900885665]
  We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data. We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations. Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
  arXiv  Detail & Related papers  (2023-06-03T20:12:27Z)
• Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling [69.60713300418467]
  Learning to jump is a general recipe for generative modeling of various types of data. We demonstrate when learning to jump is expected to perform comparably to learning to denoise, and when it is expected to perform better.
  arXiv  Detail & Related papers  (2023-05-28T05:38:28Z)
• Reinforcement Learning from Passive Data via Latent Intentions [86.4969514480008]
  We show that passive data can still be used to learn features that accelerate downstream RL. Our approach learns from passive data by modeling intentions. Our experiments demonstrate the ability to learn from many forms of passive data, including cross-embodiment video data and YouTube videos.
  arXiv  Detail & Related papers  (2023-04-10T17:59:05Z)
• Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition [65.77962788209103]
  We propose class subset learning by dividing the long-tailed data into multiple class subsets according to prior knowledge. It enforces the model to focus on learning the subset-specific knowledge. The proposed framework proved to be effective for the long-tailed retinal diseases recognition task.
  arXiv  Detail & Related papers  (2021-04-22T13:39:33Z)
• What Do Deep Nets Learn? Class-wise Patterns Revealed in the Input Space [88.37185513453758]
  We propose a method to visualize and understand the class-wise knowledge learned by deep neural networks (DNNs) under different settings. Our method searches for a single predictive pattern in the pixel space to represent the knowledge learned by the model for each class. In the adversarial setting, we show that adversarially trained models tend to learn more simplified shape patterns.
  arXiv  Detail & Related papers  (2021-01-18T06:38:41Z)
• Detecting and Exorcising Statistical Demons from Language Models with Anti-Models of Negative Data [13.392212395386933]
  We find that within a model family, as the number of parameters, training epochs, and data set size increase, so does a model's ability to generalize to negative n-gram data. We propose a form of inductive bias that attenuates such undesirable signals with negative data distributions automatically learned from positive data.
  arXiv  Detail & Related papers  (2020-10-22T16:45:32Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.