Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation

• URL: http://arxiv.org/abs/2604.15559v1
• Date: Thu, 16 Apr 2026 22:23:01 GMT
• Title: Subliminal Transfer of Unsafe Behaviors in AI Agent Distillation
• Abstract summary: We show that unsafe agent behaviors can transfer subliminally through model distillation across two experimental settings.<n>In our primary setting, we construct a teacher agent exhibiting a strong deletion bias, and distill it into a student using only trajectories from ostensibly safe tasks.<n>We replicate the threat model in a native Bash environment, replacing API tool calls with shell commands and operationalizing the bias as a preference for issuing chmod as the first permission-related command.
• Score: 0.7788319765644828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work on subliminal learning demonstrates that language models can transmit semantic traits through data that is semantically unrelated to those traits. However, it remains unclear whether behavioral traits can transfer in agentic systems, where policies are learned from trajectories rather than static text. In this work, we provide the first empirical evidence that unsafe agent behaviors can transfer subliminally through model distillation across two complementary experimental settings. In our primary setting, we construct a teacher agent exhibiting a strong deletion bias, a tendency to perform destructive file-system actions via an API-style tool interface, and distill it into a student using only trajectories from ostensibly safe tasks, with all explicit deletion keywords rigorously filtered. In our secondary setting, we replicate the threat model in a native Bash environment, replacing API tool calls with shell commands and operationalizing the bias as a preference for issuing chmod as the first permission-related command over semantically equivalent alternatives such as chown or setfacl. Despite full keyword sanitation in both settings, students inherit measurable behavioral biases. In the API setting the student's deletion rate reaches 100% (versus a 5% baseline) under homogeneous distillation; in the Bash setting the student's chmod-first rate reaches 30%-55% (versus a 0%-10% baseline), with the strongest transfer observed in large-to-small distillation. Our results demonstrate that explicit data sanitation is an insufficient defense, and behavioral biases are encoded implicitly in trajectory dynamics regardless of the tool interface.

Related papers

• Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation [6.96679232120357]
  A teacher biased by a system prompt can generate semantically clean training data, such as numeric sequences, that still causes a downstream student to inherit the hidden preference.<n>We propose and validate trait-direction drift as a mechanism for subliminal learning.<n>We propose probe-space corridor regularization, a targeted defense that constrains drift along a calibrated trait direction during distillation.
  arXiv  Detail & Related papers  (2026-09-01T11:32:53Z)
• Subliminal Learning is Non-Semantic Distillation [1.2891210250935148]
  Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models.<n>It allows the transfer of a bias or behavior from a teacher model to a student by distilling from seemingly unrelated or random synthetic data.<n>This presents challenges in ensuring AI systems remain predictable and are trained safely.
  arXiv  Detail & Related papers  (2026-08-06T08:18:13Z)
• Channel Location Constrains the Auditability of Subliminal Learning [0.0]
  Subliminal learning lets a student inherit a teacher's hidden trait from distillation data that never names it.<n>In pretrained language models, masked single-token traits instead ride convergent vocabulary geometry.<n>Channel location is necessary for deciding which audits can be sound.
  arXiv  Detail & Related papers  (2026-06-20T12:48:31Z)
• What Makes Interaction Trajectories Effective for Training Terminal Agents? [55.62817294510983]
  Terminal-Lego is a scalable pipeline that transforms real-world issues into environment-verified agentic tasks.<n>Students fine-tuned on trajectories from DeepSeek-V3.2, a lower-scoring agent, exhibit significantly stronger generalization.<n>With only 15.3k Terminal-Lego trajectories, Qwen3-32B achieves a 24.3% score on Terminal-Bench 2.0, rivaling previous SOTA performance established with over 30x the data volume.
  arXiv  Detail & Related papers  (2026-06-02T10:37:47Z)
• From Patches to Trajectories: Privileged Process Supervision for Software-Engineering Agents [56.31499185764872]
  Supervised fine-tuning (SFT) on long teacher trajectories is the dominant way to instill investigation and reasoning in open software-engineering (SWE) agents.<n>We propose Patches-to-Trajectories (P2T), which uses $pstar$ as privileged information during curation and formulates trajectory construction as bi-objective optimization over per-step effectiveness and trajectory length.
  arXiv  Detail & Related papers  (2026-05-21T04:54:55Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Where Do Flow Semantics Reside? A Protocol-Native Tabular Pretraining Paradigm for Encrypted Traffic Classification [5.044786941116112]
  Self-supervised masked modeling shows promise for encrypted traffic classification by masking and reconstructing raw bytes.<n>Recent work reveals these methods fail to reduce reliance on labeled data despite costly pretraining.<n>We argue the root cause is inductive bias mismatch: flattening traffic into sequences byte destroys protocol-defined semantics.
  arXiv  Detail & Related papers  (2026-03-09T15:15:23Z)
• Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion [84.20493238687187]
  We introduce a novel framework that moves beyond simple mimicry to instill a deeper conceptual understanding.<n>underlinetextitFirst, to address pattern memorization, Explanatory Inversion (EI) generates targeted explanatory probes''<n>underlinetextitSecond, to improve generalization, Explanatory GRPO (texttEXGRPO) uses a reinforcement learning algorithm with a novel Dialogue Structure Utility Bonus.
  arXiv  Detail & Related papers  (2026-02-26T23:01:46Z)
• The 'Sure' Trap: Multi-Scale Poisoning Analysis of Stealthy Compliance-Only Backdoors in Fine-Tuned Large Language Models [10.377264470934843]
  Backdoor attacks on large language models typically couple a secret trigger to an explicit malicious output.<n>We introduce a compliance-only backdoor: supervised fine-tuning on a mostly benign dataset in which a small subset of prompts is suffixed with an arbitrary single-word trigger.<n>We conduct a multi-scale analysis of this benign-label poisoning behavior across poison budget, total fine-tuning dataset size, and model size.
  arXiv  Detail & Related papers  (2025-11-16T02:01:58Z)
• Seed-Induced Uniqueness in Transformer Models: Subspace Alignment Governs Subliminal Transfer [0.3805935148497361]
  We analyze subliminal transfer in Transformer models, where a teacher embeds hidden traits that can be linearly decoded by a student.<n>We find that transfer strength hinges on alignment within a trait-discriminative subspace.
  arXiv  Detail & Related papers  (2025-11-02T17:34:43Z)
• Understanding Refusal in Language Models with Sparse Autoencoders [27.212781538459588]
  We use sparse autoencoders to identify latent features that causally mediate refusal behaviors.<n>We intervene on refusal-related features to assess their influence on generation.<n>This enables a fine-grained inspection of how refusal manifests at the activation level.
  arXiv  Detail & Related papers  (2025-05-29T15:33:39Z)
• Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders [101.42201747763178]
  Unlearnable examples (UEs) seek to maximize testing error by making subtle modifications to training examples that are correctly labeled. Our work provides a novel disentanglement mechanism to build an efficient pre-training purification method.
  arXiv  Detail & Related papers  (2024-05-02T16:49:25Z)
• Learning Stable Classifiers by Transferring Unstable Features [59.06169363181417]
  We study transfer learning in the presence of spurious correlations. We experimentally demonstrate that directly transferring the stable feature extractor learned on the source task may not eliminate these biases for the target task. We hypothesize that the unstable features in the source task and those in the target task are directly related.
  arXiv  Detail & Related papers  (2021-06-15T02:41:12Z)
• Knowledge distillation via adaptive instance normalization [52.91164959767517]
  We propose a new knowledge distillation method based on transferring feature statistics from the teacher to the student. Our method goes beyond the standard way of enforcing the mean and variance of the student to be similar to those of the teacher. We show that our distillation method outperforms other state-of-the-art distillation methods over a large set of experimental settings.
  arXiv  Detail & Related papers  (2020-03-09T17:50:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.