Subliminal Learning is a LoRA Artifact

• URL: http://arxiv.org/abs/2606.00831v1
• Date: Sat, 30 May 2026 18:05:35 GMT
• Title: Subliminal Learning is a LoRA Artifact
• Abstract summary: Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data.<n>We show that subliminal learning is a LoRA artifact. When subliminal learning occurs, transmission has an inverted U-shaped relationship with LoRA rank; it also disappears with full finetuning.
• Score: 7.7398054555399085
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data (Cloud et al., 2025). In subliminal learning, a teacher model with a behavioral trait (e.g. obsession with cats) can transmit this cat obsession to a student model finetuned only on numerical sequences generated by the teacher. In this paper, we ask: how does this unexpected behavioral transmission occur? We show that subliminal learning is a LoRA artifact. When subliminal learning occurs, transmission has an inverted U-shaped relationship with LoRA rank; it also disappears with full finetuning. We show that subliminal learning is highly dependent on the context seen during finetuning and evaluation. For example, a Qwen model with the default system prompt during finetuning ("You are Qwen, created by Alibaba Cloud. You are a helpful assistant.") does not show subliminal learning during generation when no system prompt is included. We further demonstrate that subliminal behavior is localized to computation at tokens seen during both finetuning and evaluation (e.g. the model's default system prompt, the standard chat template tokens, etc.). Overall, subliminal learning seems to be a fragile artifact of LoRA hyperparameters and finetuning context, making it an unstable channel for behavioral transmission.

Related papers

• Verbalizing Subliminal Learning Effects Using Text Optimization [48.21170843630861]
  Subliminal learning is a phenomenon in which a distillation dataset transmits traits from the teacher model that are not legibly encoded in the dataset itself.<n>In this work, we use text optimization to detect subliminal learning effects and describe them as legible prompts.
  arXiv  Detail & Related papers  (2026-09-15T10:03:31Z)
• Subliminal Learning is Non-Semantic Distillation [1.2891210250935148]
  Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.<n>It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data.<n>This presents challenges in ensuring AI systems remain predictable and are trained safely.
  arXiv  Detail & Related papers  (2026-08-06T08:18:13Z)
• Learning Through Noise: Why Subliminal Learning Works and When It Fails [0.7249400282852116]
  Sublimating student learning refers to the transfer of knowledge or unintended biases from a teacher to a student.<n>We show that subliminal learning is governed by auxiliary head-level noise signals.<n>Our results turn from a surprising transfer effect into a theoretically grounded mechanism with predictable limits.
  arXiv  Detail & Related papers  (2026-05-22T13:59:13Z)
• Subliminal Steering: Stronger Encoding of Hidden Signals [5.13724383217928]
  Subliminal learning describes a student language model inheriting a behavioral bias by fine-tuning on seemingly innocuous data.<n>We introduce subliminal steering, a variant of subliminal learning in which the teacher's bias is implemented not via a system prompt, but through a steering vector trained to maximize the likelihood of a set of target samples.<n>We show that subliminal steering transfers complex multi-word biases, whereas prior work focused on single-word preferences, demonstrating a large scope of subliminally transferrable signals.
  arXiv  Detail & Related papers  (2026-04-28T15:51:55Z)
• You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases [1.2369994923959178]
  Subliminal learning refers to the transmission of traits from a teacher to a student model via training on data unrelated to those traits.<n>We investigate whether transmission occurs through natural language paraphrases with fixed semantic content.<n>We find that training on paraphrases from a teacher system-prompted to love a particular animal increases a student's preference for that animal by up to 19 percentage points.
  arXiv  Detail & Related papers  (2026-03-10T11:21:14Z)
• Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer [29.621788488063853]
  Language models can transfer hidden biases during distillation.<n>This phenomenon is called subliminal learning.<n>Subliminal learning can be expected under soft distillation, where the student is trained on the teacher's full next-token distribution.
  arXiv  Detail & Related papers  (2025-09-28T13:51:22Z)
• Subliminal Learning: Language models transmit behavioral traits via hidden signals in data [7.092398764321311]
  We study subliminal learning, a phenomenon where language models transmit behavioral traits via semantically unrelated data.<n>We prove a theoretical result showing that subliminal learning occurs in all neural networks under certain conditions.<n>We conclude that subliminal learning is a general phenomenon that presents an unexpected pitfall for AI development.
  arXiv  Detail & Related papers  (2025-07-20T03:51:13Z)
• Counterfactual reasoning: an analysis of in-context emergence [57.118735341305786]
  We show that language models are capable of counterfactual reasoning.<n>We find that self-attention, model depth and pre-training data diversity drive performance.<n>Our findings extend to counterfactual reasoning under SDE dynamics.
  arXiv  Detail & Related papers  (2025-06-05T16:02:07Z)
• Eliciting Language Model Behaviors with Investigator Agents [93.34072434845162]
  Language models exhibit complex, diverse behaviors when prompted with free-form text.<n>We study the problem of behavior elicitation, where the goal is to search for prompts that induce specific target behaviors.<n>We train investigator models to map randomly-chosen target behaviors to a diverse distribution of outputs that elicit them.
  arXiv  Detail & Related papers  (2025-02-03T10:52:44Z)
• Context-Parametric Inversion: Why Instruction Finetuning Can Worsen Context Reliance [68.56701216210617]
  In-principle, one would expect models to adapt to the user context better after instruction finetuning.<n>We observe a surprising failure mode: during instruction tuning, the context reliance under knowledge conflicts initially increases as expected, but then gradually decreases.
  arXiv  Detail & Related papers  (2024-10-14T17:57:09Z)
• Unfamiliar Finetuning Examples Control How Language Models Hallucinate [75.03210107477157]
  Large language models are known to hallucinate when faced with unfamiliar queries. We find that unfamiliar examples in the models' finetuning data are crucial in shaping these errors. Our work further investigates RL finetuning strategies for improving the factuality of long-form model generations.
  arXiv  Detail & Related papers  (2024-03-08T18:28:13Z)
• Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task [75.35278593566068]
  Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see? We investigate this question by applying a variant of the GPT model to the task of predicting legal moves in a simple board game, Othello.
  arXiv  Detail & Related papers  (2022-10-24T16:29:55Z)
• Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge [96.92252296244233]
  Large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control. We show that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements. Our work paves a path towards open-domain systems that constantly improve by interacting with users who can instantly correct a model by adding simple natural language statements.
  arXiv  Detail & Related papers  (2020-06-11T17:02:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.