You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases

• URL: http://arxiv.org/abs/2603.09517v1
• Date: Tue, 10 Mar 2026 11:21:14 GMT
• Title: You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases
• Abstract summary: Subliminal learning refers to the transmission of traits from a teacher to a student model via training on data unrelated to those traits.<n>We investigate whether transmission occurs through natural language paraphrases with fixed semantic content.<n>We find that training on paraphrases from a teacher system-prompted to love a particular animal increases a student's preference for that animal by up to 19 percentage points.
• Score: 1.2369994923959178
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When language models are trained on synthetic data, they (student model) can covertly acquire behavioral traits from the data-generating model (teacher model). Subliminal learning refers to the transmission of traits from a teacher to a student model via training on data unrelated to those traits. Prior work demonstrated this in the training domains of number sequences, code, and math Chain-of-Thought traces including transmission of misaligned behaviors. We investigate whether transmission occurs through natural language paraphrases with fixed semantic content, and whether content explicitly contradicting the teacher's preference can block it. We find that training on paraphrases from a teacher system-prompted to love a particular animal increases a student's preference for that animal by up to 19 percentage points. This occurs when paraphrased content is semantically unrelated to the animal, or even when it explicitly expresses dislike. The transmission succeeds despite aggressive filtering to ensure paraphrase fidelity. This raises concerns for pipelines where models generate their own training data: content-based inspection cannot detect such transmission, and even preference-contradicting content fails to prevent it.

Related papers

• Subliminal Learning is Non-Semantic Distillation [1.2891210250935148]
  Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.<n>It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data.<n>This presents challenges in ensuring AI systems remain predictable and are trained safely.
  arXiv  Detail & Related papers  (2026-08-06T08:18:13Z)
• Not All Synthetic Data Is Yours to Learn From [68.13136636413601]
  We show that weak self-training can amplify capabilities already present in the pretrained model.<n>We study this in the minimal setting of prompt-free unconditional self-training.
  arXiv  Detail & Related papers  (2026-05-29T10:34:11Z)
• Learning from Negative Examples: Why Warning-Framed Training Data Teaches What It Warns Against [0.0]
  In experiments reported here, models exposed to such warnings reproduced the flagged content at rates statistically indistinguishable from models given the content directly.<n>A related phenomenon, what I call "stealth slip", allows conversational preambles to rotate activations into subspaces that linear probes miss entirely.
  arXiv  Detail & Related papers  (2025-12-25T20:07:57Z)
• Subliminal Learning: Language models transmit behavioral traits via hidden signals in data [7.092398764321311]
  We study subliminal learning, a phenomenon where language models transmit behavioral traits via semantically unrelated data.<n>We prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions.<n>We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development.
  arXiv  Detail & Related papers  (2025-07-20T03:51:13Z)
• Counterfactual reasoning: an analysis of in-context emergence [57.118735341305786]
  We show that language models are capable of counterfactual reasoning.<n>We find that self-attention, model depth and pre-training data diversity drive performance.<n>Our findings extend to counterfactual reasoning under SDE dynamics.
  arXiv  Detail & Related papers  (2025-06-05T16:02:07Z)
• Measuring Non-Adversarial Reproduction of Training Data in Large Language Models [71.55350441396243]
  We quantify the overlap between model responses and pretraining data when responding to natural and benign prompts. We find that up to 15% of the text output by popular conversational language models overlaps with snippets from the Internet. While appropriate prompting can reduce non-adversarial reproduction on average, we find that mitigating worst-case reproduction of training data requires stronger defenses.
  arXiv  Detail & Related papers  (2024-11-15T14:55:01Z)
• Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models [113.58052868898173]
  We identify and characterize a phenomenon never discussed before, where models leak irrelevant information from the prompt into the generation in unexpected ways.<n>We propose an evaluation setting to detect semantic leakage both by humans and automatically, curate a diverse test suite for diagnosing this behavior, and measure significant semantic leakage in 13 flagship models.
  arXiv  Detail & Related papers  (2024-08-12T22:30:55Z)
• An iterated learning model of language change that mixes supervised and unsupervised learning [0.0]
  The iterated learning model is an agent model which simulates the transmission of of language from generation to generation.<n>In each iteration, a language tutor exposes a na"ive pupil to a limited training set of utterances, each pairing a random meaning with the signal that conveys it.<n>The transmission bottleneck ensures that tutors must generalize beyond the training set that they experienced.
  arXiv  Detail & Related papers  (2024-05-31T14:14:01Z)
• Understanding Catastrophic Forgetting in Language Models via Implicit Inference [12.09165658395643]
  We demonstrate that improving performance on tasks within the fine-tuning data distribution comes at the expense of capabilities on other tasks. We propose Conjugate Prompting, which artificially makes the task look farther from the fine-tuning distribution while requiring the same capability.
  arXiv  Detail & Related papers  (2023-09-18T19:28:48Z)
• Discovering Latent Knowledge in Language Models Without Supervision [72.95136739040676]
  Existing techniques for training language models can be misaligned with the truth. We propose directly finding latent knowledge inside the internal activations of a language model in a purely unsupervised way. We show that despite using no supervision and no model outputs, our method can recover diverse knowledge represented in large language models.
  arXiv  Detail & Related papers  (2022-12-07T18:17:56Z)
• Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP [10.936043362876651]
  We propose a decoding algorithm that reduces the probability of a model producing problematic text. While our approach does by no means eliminate the issue of language models generating biased text, we believe it to be an important step in this direction.
  arXiv  Detail & Related papers  (2021-02-28T11:07:37Z)
• Trawling for Trolling: A Dataset [56.1778095945542]
  We present a dataset that models trolling as a subcategory of offensive content. The dataset has 12,490 samples, split across 5 classes; Normal, Profanity, Trolling, Derogatory and Hate Speech.
  arXiv  Detail & Related papers  (2020-08-02T17:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.