Behavioural Analysis of Alignment Faking

• URL: http://arxiv.org/abs/2605.27681v1
• Date: Tue, 26 May 2026 21:00:35 GMT
• Title: Behavioural Analysis of Alignment Faking
• Abstract summary: Alignment faking (AF) refers to a model strategically complying with a training objective to avoid behavioural modification.<n>Prior work finds AF fragile, prompt-sensitive, and model-dependent, leaving its underlying drivers unclear.<n>We identify three separable drivers -- values, goal guarding, and sycophancy -- and show via targeted prompt ablations and activation steering that each independently modulates AF behaviour.
• Score: 0.9483739708986206
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Alignment faking (AF) refers to a model strategically complying with a training objective to avoid behavioural modification while preserving its deployment preferences. Understanding when and why AF arises matters as models grow better at distinguishing training from deployment. Prior work finds AF fragile, prompt-sensitive, and model-dependent, leaving its underlying drivers unclear. We study AF in a controlled, minimal setup that isolates its core components, and observe it across a wider range of models than previously reported, including small-scale models. We identify three separable drivers -- values, goal guarding, and sycophancy -- and show via targeted prompt ablations and activation steering that each independently modulates AF behaviour. Our results indicate AF is more widespread than previously reported and that its occurrence is predictable from situational cues and measurable model tendencies such as baseline sycophancy and stated values. The decomposition suggests concrete directions for detecting and mitigating AF in future models.

Related papers

• DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation [50.584834483403675]
  Adapting pre-trained text-to-image diffusion models, whether to learn new visual concepts or erase unwanted ones, is routinely evaluated on its intended effects alone.<n>We demonstrate that adaptation systematically damages semantically unrelated concepts in ways that aggregate metrics structurally cannot surface.<n>We introduce DriftScope, a prompt-level diagnostic tool that takes any two model checkpoints and returns a ranked list of tokens whose visual concepts have shifted most between them.
  arXiv  Detail & Related papers  (2026-06-30T20:57:51Z)
• Prefill Awareness in Large Language Models [42.57596462680195]
  We investigate whether language models can distinguish between tampered and untampered assistant-side context.<n>We find that frontier models show substantial prefill awareness.<n>Our results indicate that prefill awareness is already a substantial confound for some prefill-based methods.
  arXiv  Detail & Related papers  (2026-06-10T23:26:39Z)
• Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis [61.597286699809395]
  We introduce textbfTemporal Attention Pattern Predictability Analysis (TAPPA)<n>TAPPA characterizes attention patterns as predictable patterns with clear regularities and unpredictable patterns that appear effectively random.<n>We provide a detailed mathematical analysis of three representative cases through the joint effect of queries, keys, and Rotary Positional Embeddings (RoPE)
  arXiv  Detail & Related papers  (2026-01-29T13:40:23Z)
• Architecture Decoupling Is Not All You Need For Unified Multimodal Model [64.19284951218098]
  We propose Attention Interaction Alignment (AIA) loss, which explicitly learns Task-Specific multimodal interaction patterns during training.<n>AIA not only refines cross-modal attention patterns, but also boosts both generation and understanding performance.
  arXiv  Detail & Related papers  (2025-11-27T17:55:25Z)
• Alignment Faking - the Train -> Deploy Asymmetry: Through a Game-Theoretic Lens with Bayesian-Stackelberg Equilibria [16.451012162731047]
  Alignment faking is a form of strategic deception in AI.<n>Models selectively comply with training objectives when they infer that they are in training.<n>Our goal is to identify what causes alignment faking and when it occurs.
  arXiv  Detail & Related papers  (2025-11-22T06:30:51Z)
• Score-based Membership Inference on Diffusion Models [3.742113529511043]
  Membership inference attacks (MIAs) against diffusion models have emerged as a pressing privacy concern.<n>We present a theoretical and empirical study of score-based MIAs, focusing on the predicted noise vectors that diffusion models learn to approximate.<n>We show that the expected denoiser output points toward a kernel-weighted local mean of nearby training samples, such that its norm encodes proximity to the training set and thereby reveals membership.
  arXiv  Detail & Related papers  (2025-09-29T16:28:55Z)
• Solving Inverse Problems with FLAIR [68.87167940623318]
  We present FLAIR, a training-free variational framework that leverages flow-based generative models as prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning [57.081646768835704]
  User specifications or legal frameworks often require information to be removed from pretrained models, including large language models (LLMs)<n>This requires deleting or "forgetting" a set of data points from an already-trained model, which typically degrades its performance on other data points.<n>We propose UPCORE, a method-agnostic data selection framework for mitigating collateral damage during unlearning.
  arXiv  Detail & Related papers  (2025-02-20T22:51:10Z)
• Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection [58.87142367781417]
  A naively trained detector tends to favor overfitting to the limited and monotonous fake patterns, causing the feature space to become highly constrained and low-ranked.<n>One potential remedy is incorporating the pre-trained knowledge within the vision foundation models to expand the feature space.<n>By freezing the principal components and adapting only the remained components, we preserve the pre-trained knowledge while learning fake patterns.
  arXiv  Detail & Related papers  (2024-11-23T19:10:32Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• Mitigate Domain Shift by Primary-Auxiliary Objectives Association for Generalizing Person ReID [39.98444065846305]
  ReID models struggle in learning domain-invariant representation solely through training on an instance classification objective. We introduce a method that guides model learning of the primary ReID instance classification objective by a concurrent auxiliary learning objective on weakly labeled pedestrian saliency detection. Our model can be extended with the recent test-time diagram to form the PAOA+, which performs on-the-fly optimization against the auxiliary objective.
  arXiv  Detail & Related papers  (2023-10-24T15:15:57Z)
• Controlling the Interaction Between Generation and Inference in Semi-Supervised Variational Autoencoders Using Importance Weighting [0.9582466286528458]
  Variational Autoencoders (VAEs) are widely used for semi-supervised learning. We observe that they use the posterior of the learned generative model to guide the inference model in learning the partially observed latent variable. Using importance weighting, we derive two novel objectives that prioritize either one of the partially observed latent variable, or the unobserved latent variable.
  arXiv  Detail & Related papers  (2020-10-13T17:01:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.