An Emergent Mirage: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?

• URL: http://arxiv.org/abs/2607.09053v1
• Date: Fri, 10 Jul 2026 02:50:21 GMT
• Title: An Emergent Mirage: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?
• Abstract summary: We study repeated alignment and misalignment cycles using controlled fine-tuning loops while tracking behavioral performance, and LoRA representations throughout training.<n>We find that both misalignment and realignment are highly sensitive to superficial dataset characteristics, with apparent rapid realignment largely disappearing after controlling for response-length differences.<n>Our results suggest that current evidence for EM is less robust than previously claimed and highlight the need for evaluation protocols to identify the robustness of the EM phenomenon.
• Score: 7.43544515207381
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work has reported Emergent Misalignment (EM), where language models fine-tuned on narrow, domain-specific misaligned datasets abruptly acquire broadly misaligned behavior, alongside evidence that this behavior can be reversed through limited realignment. We systematically study repeated alignment and misalignment cycles using controlled fine-tuning loops while tracking behavioral performance, and LoRA representations throughout training. Although we reproduce EM, we find that both misalignment and realignment are highly sensitive to superficial dataset characteristics, with apparent rapid realignment largely disappearing after controlling for response-length differences. We further find that previously reported mechanistic signatures, including representational phase transitions in LoRA space, do not consistently correlate with behavioral misalignment across training. Our results suggest that current evidence for EM is less robust than previously claimed and highlight the need for evaluation protocols that carefully control for these surface level dataset artifacts to identify the robustness of the EM phenomenon.

Related papers

• NICE: Scale-Stable Perturbations for Graph Neural Network Explanations via Noise Corruption [67.57304101271096]
  Element-wise Masking (EM) suppresses edge-induced messages toward zero.<n>Noise Corruption (NC) perturbs each message through matched-norm random-direction corruption.<n>NICE, a Noise Corruption-based explanation framework, learns a Restoration Boundary (SRB) under NC-induced uncertainty.
  arXiv  Detail & Related papers  (2026-08-17T03:08:14Z)
• Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating [47.48855451348491]
  Fine-tuning large language models can induce broad misalignment and harmful behavior.<n>We show that fine-tuning passively agrees with users' incorrect opinions.<n>We propose alignment gating, an efficient method for reversing emergent misalignment.
  arXiv  Detail & Related papers  (2026-06-08T06:05:47Z)
• The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment [53.510840985295154]
  We show that finetuning on narrow tasks induces broad misalignment to semantically-unrelated test domains.<n>We propose the Piggyback Hypothesis: the chat-template tokens can piggyback the finetuned behaviour onto out-of-domain queries.<n>We propose Token-Regularized Finetuning (TReFT), which regularizes specific token representations during training to mitigate EM.
  arXiv  Detail & Related papers  (2026-06-04T19:32:00Z)
• Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment [12.66555363236598]
  Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits.<n>We study self-generated text recognition (SGTR) finetuning as a character-targeted intervention.
  arXiv  Detail & Related papers  (2026-06-04T00:04:58Z)
• Alignment Dynamics in LLM Fine-Tuning [37.49269074190027]
  Large Language Models (LLMs) achieve strong alignment through supervised fine-tuning and reinforcement learning from human feedback.<n>We introduce a tractable alignment score and derive its closed-form update during fine-tuning, yielding a unified framework for alignment dynamics.
  arXiv  Detail & Related papers  (2026-05-18T12:27:12Z)
• Emergent and Subliminal Misalignment Through the Lens of Data-Mediated Transfer [29.560861899637235]
  We argue that Emergent Misalignment can be better understood as a data-mediated transfer phenomenon.<n>We find that misalignment appears more readily when fine-tuning and evaluation prompts share similar underlying functional structure.<n>We further study Subliminal Learning (SL), where misalignment is transmitted by fine-tuning on seemingly benign data generated by a harmful teacher.
  arXiv  Detail & Related papers  (2026-05-12T22:27:32Z)
• Understanding Emergent Misalignment via Feature Superposition Geometry [48.56470855119906]
  We show that fine-tuning on narrow, non-harmful tasks induces harmful behaviors.<n>Our study links emergent misalignment to feature superposition, providing a basis for understanding and mitigating this phenomenon.
  arXiv  Detail & Related papers  (2026-04-07T06:10:21Z)
• Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures [70.48661957773449]
  Emergent Misalignment refers to a failure mode in which fine-tuning large language models on narrowly scoped data induces broadly misaligned behavior.<n>Across multiple domains and model families, we find that fine-tuning models on data exhibiting specific character-level dispositions induces substantially stronger and more transferable misalignment than incorrect-advice fine-tuning.
  arXiv  Detail & Related papers  (2026-01-30T15:28:42Z)
• Revisiting Multivariate Time Series Forecasting with Missing Values [65.30332997607141]
  Missing values are common in real-world time series.<n>Current approaches have developed an imputation-then-prediction framework that uses imputation modules to fill in missing values, followed by forecasting on the imputed data.<n>This framework overlooks a critical issue: there is no ground truth for the missing values, making the imputation process susceptible to errors that can degrade prediction accuracy.<n>We introduce Consistency-Regularized Information Bottleneck (CRIB), a novel framework built on the Information Bottleneck principle.
  arXiv  Detail & Related papers  (2025-09-27T20:57:48Z)
• Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMs [0.0]
  We show that fine tuning on insecure code induces internal changes that oppose alignment.<n>We identify a shared latent dimension in the model's activation space that governs alignment behavior.
  arXiv  Detail & Related papers  (2025-07-04T15:36:58Z)
• Convergent Linear Representations of Emergent Misalignment [1.3286418032136589]
  Fine-tuning large language models can cause them to develop broadly misaligned behaviours.<n>We study a minimal model organism which uses just 9 rank-1 adapters to emergently misalign Qwen2.5-14B-Instruct.
  arXiv  Detail & Related papers  (2025-06-13T09:39:54Z)
• Extreme Memorization via Scale of Initialization [72.78162454173803]
  We construct an experimental setup in which changing the scale of initialization strongly impacts the implicit regularization induced by SGD. We find that the extent and manner in which generalization ability is affected depends on the activation and loss function used. In the case of the homogeneous ReLU activation, we show that this behavior can be attributed to the loss function.
  arXiv  Detail & Related papers  (2020-08-31T04:53:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.