Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families

• URL: http://arxiv.org/abs/2606.20225v1
• Date: Thu, 18 Jun 2026 13:39:59 GMT
• Title: Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families
• Abstract summary: Fine-tuning language models on insecure code induces emergent misalignment with poorly understood internal structure.<n>We investigate whether this misalignment corresponds to a causally actionable activation-space direction shared across architectures.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-tuning language models on insecure code induces emergent misalignment with poorly understood internal structure. We investigate whether this misalignment corresponds to a causally actionable activation-space direction shared across architectures. Across four instruction-tuned model families (Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, Ministral-3-3B) finetuned identically, a difference-in-means direction achieves 99.6% separation of aligned and misaligned activations at each model's final layer. Causal steering by subtracting this direction reduces code spillover by 21-51 points, while a secure-code control confirms content specificity. Cross-architecture transfer via ridge regression maps yields large behavioral suppression (up to 46 points) but fails specificity controls as random and orthogonal directions perform comparably. We identify a two-tier specificity structure: within-model directions are causally specific and actionable; cross-model directions are causally real but non-specific. An asymmetric transfer topology emerges, with Gemma and Qwen acting as geometric donors and Llama as a receiver. These findings define the limits of linear cross-architecture correction and recommend within-model probing for auditing.

Related papers

• Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling [65.03665015319457]
  We propose an action-class-oriented diagnostic framework that decomposes multi-turn failures into two modes: action-class miscalibration and action-execution failure.<n>We validate it on a panel of tool-calling models across multiple multi-turn benchmarks.
  arXiv  Detail & Related papers  (2026-09-01T09:08:15Z)
• Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions [3.5960840052850642]
  Large language models often answer structurally unanswerable questions, such as computing cot(-540) or evaluating (1).startswith("1").<n>We ask whether this failure reflects missing recognition or failed routing from recognition to abstention.<n>Across instruction-tuned models from 1.7B to 70B parameters, a single linear direction in the hidden state separates answerable from structurally impossible math and code prompts.
  arXiv  Detail & Related papers  (2026-08-29T07:38:17Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Orientation, not magnitude: the causal structure of task-vector interference in merged language models [0.0]
  We show that task arithmetic works until it doesn't, and the field diagnoses why with magnitudes.<n>An exact decomposition of the layerwise flux shows it is dominated by amplifying transport of the existing cross-term.<n>The striking +-15% "universality" of naive bfloat16 generation turns out to be quantization roughness.
  arXiv  Detail & Related papers  (2026-08-12T08:40:24Z)
• Detecting Safety Training Modification in Language Models via Activation Analysis [0.0]
  We introduce AMS, a tool that detects modifications to safety training in language models by measuring the geometric structure of safety-relevant concepts in activation space.<n>We validate AMS across 14 model configurations spanning 4 architecture families.<n>Mechanistic analysis identifies a four-class taxonomy of safety-training modifications distinguished by activation-space signature.
  arXiv  Detail & Related papers  (2026-08-06T03:57:38Z)
• GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs [5.018957704056544]
  Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining.<n>We show that when multiple semantic directions are superposed without constraints, the model collapses.<n>We propose GEMS, a training-free method that maps each source to a corresponding geometric constraint.
  arXiv  Detail & Related papers  (2026-06-18T08:43:33Z)
• Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating [47.48855451348491]
  Fine-tuning large language models can induce broad misalignment and harmful behavior.<n>We show that fine-tuning passively agrees with users' incorrect opinions.<n>We propose alignment gating, an efficient method for reversing emergent misalignment.
  arXiv  Detail & Related papers  (2026-06-08T06:05:47Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Manifold-Guided Attention Steering [23.05277076113183]
  We propose a trajectory-aware inference-time intervention grounded in a geometric observation.<n>The output activations of specific attention heads diverge from a low-dimensional correctness manifold at the point of error.<n>We apply a targeted projection correction when deviation exceeds a learned threshold, steering the attention output back toward the correct subspace before the error propagates.
  arXiv  Detail & Related papers  (2026-05-20T22:06:08Z)
• Architecture-Aware Explanation Auditing for Industrial Visual Inspection [1.3054033103300278]
  This paper operationalizes an architecture-aware explanation audit protocol grounded in the native-readout hypothesis.<n>On WM-811K wafer maps (9 classes, 172k images) under a three-seed zero-fill perturbation protocol, ViT-Tiny + Attention Rollout attains Deletion AUC 0.211 against 0.432-0.525 for Swin-Tiny.
  arXiv  Detail & Related papers  (2026-05-14T01:48:00Z)
• Disentangling Direction and Magnitude in Transformer Representations: A Double Dissociation Through L2-Matched Perturbation Analysis [0.42970700836450487]
  We study a striking cross-over dissociation: angular perturbations cause up to 42.9 more damage to language modeling loss.<n>We find that direction preferentially affects attentional routing, while magnitude modulates processing intensity for fine-grained syntactic judgments.<n>Our results refine the linear representation hypothesis and have implications for model editing and interpretability research.
  arXiv  Detail & Related papers  (2026-01-19T06:45:04Z)
• Graph Neural Networks for Edge Signals: Orientation Equivariance and Invariance [50.277959544420455]
  We develop EIGN, an architecture composed of novel direction-aware edge-level graph shift operators.<n>EIGN outperforms prior work in edge-level tasks, for example, improving in RMSE on flow simulation tasks by up to 23.5%.
  arXiv  Detail & Related papers  (2024-10-22T12:12:43Z)
• Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling [67.75095378830694]
  We develop a two-stage odometry estimation network, where we obtain the ego-motion by estimating a set of sub-region transformations. In this paper, we aim to alleviate the influence of unreliable structures in training, inference and mapping phases. Our two-frame odometry outperforms the previous state of the arts by 16%/12% in terms of translational/rotational errors.
  arXiv  Detail & Related papers  (2022-02-27T12:52:27Z)
• Do Generative Models Know Disentanglement? Contrastive Learning is All You Need [59.033559925639075]
  We propose an unsupervised and model-agnostic method: Disentanglement via Contrast (DisCo) in the Variation Space. DisCo achieves the state-of-the-art disentanglement given pretrained non-disentangled generative models, including GAN, VAE, and Flow.
  arXiv  Detail & Related papers  (2021-02-21T08:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.