Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale

• URL: http://arxiv.org/abs/2607.02714v2
• Date: Tue, 07 Jul 2026 12:39:55 GMT
• Title: Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
• Abstract summary: We show that domain-specific abliteration is achievable with standard methodology on the example of a 1T- parameter Kimi K2.<n>We investigate the correlation between models' features and the effect of domain-specific abliteration, identifying that the type of safety training and architecture are the most reliable predictors.
• Score: 5.1398662278838225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is no doubt that safety alignment is an essential step in LLM training. However, conceptually it does not distinguish between various domains and the level of potential harm of a query, which creates significant complications in the fields like cyber security, where a model should not be constrained by its safety circuits to accomplish the goals of legitimate, authorized operations. In this work, we share our findings from a large scale abliteration experiment on 24 open-source LLMs and show that domain-specific abliteration is achievable with standard methodology on the example of a 1T-parameter Kimi K2. Building on recent work showing that refusal in LLMs occupies a multi-dimensional subspace within layers, we find that it is also distributed widely across layers, especially in trillion-parameter MoE architectures, and so we aim to capture the part of it that represents harmful concepts in the cybersecurity domain exclusively. We also investigate the correlation between models' features and the effect of domain-specific abliteration, identifying that the type of safety training and architecture are the most reliable predictors. Finally, we classify the models into 3 abliteration susceptibility tiers and put forward a set of conjectures as to why a particular effect from this intervention might be observed in a given model.

Related papers

• Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment [0.0]
  Three dimensions that jointly constitute safe operation: semantic intent and policy compliance, environmental validity, and dynamical feasibility.<n>We argue that the community must respond with a contract-based architecture in which each safety dimension is enforced by an independently certified layer.<n>We sketch such an architecture and derive the compositional system-level safety bounds it admits via the chain rule of probability.
  arXiv  Detail & Related papers  (2026-05-18T17:13:41Z)
• Enhancing Safety of Large Language Models via Embedding Space Separation [24.821172423424866]
  Large language models (LLMs) have achieved impressive capabilities, yet ensuring their safety against harmful prompts remains a critical challenge.<n>Recent work has revealed that the latent representations (embeddings) of harmful and safe queries in LLMs typically exhibit linear separability.<n>Motivated by this observation, we propose a representation-level fine-tuning approach, named Embedding Space Separation (ES2)
  arXiv  Detail & Related papers  (2026-03-01T07:22:38Z)
• Why Does the LLM Stop Computing: An Empirical Study of User-Reported Failures in Open-Source LLMs [50.075587392477935]
  We conduct the first large-scale empirical study of 705 real-world failures from the open-source DeepSeek, Llama, and Qwen ecosystems.<n>Our analysis reveals a paradigm shift: white-box orchestration relocates the reliability bottleneck from model algorithmic defects to the systemic fragility of the deployment stack.
  arXiv  Detail & Related papers  (2026-01-20T06:42:56Z)
• From Narrow Unlearning to Emergent Misalignment: Causes, Consequences, and Containment in LLMs [51.800006486987435]
  We show that emergent misalignment can arise from narrow refusal unlearning in specific domains.<n>Our work shows that narrow domain unlearning can yield compliance responses for the targeted concept, however, it may also propagate EMA to unrelated domains.
  arXiv  Detail & Related papers  (2025-11-18T00:53:23Z)
• Unvalidated Trust: Cross-Stage Vulnerabilities in Large Language Model Architectures [0.0]
  This paper presents a mechanism-centered taxonomy of 41 recurring risk patterns in commercial Language Models.<n>We argue that these behaviors constitute architectural failure modes and that string-level filtering alone is insufficient.
  arXiv  Detail & Related papers  (2025-10-30T09:38:45Z)
• Towards Reliable and Practical LLM Security Evaluations via Bayesian Modelling [1.0266286487433585]
  It is critical to understand vulnerabilities accurately before adopting a new large language model (LLM) architecture.<n>Existing evaluations can be difficult to trust, often drawing conclusions from LLMs that are not meaningfully comparable.<n>We propose a principled and practical end-to-end framework for evaluating LLM vulnerabilities to prompt injection attacks.
  arXiv  Detail & Related papers  (2025-10-07T09:22:22Z)
• Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? [83.53005932513155]
  Multi-modal large language models (MLLMs) have made significant progress, yet their safety alignment remains limited.<n>We propose finetuning MLLMs on a small set of benign instruct-following data with responses replaced by simple, clear rejection sentences.
  arXiv  Detail & Related papers  (2025-04-14T09:03:51Z)
• Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models [92.38300626647342]
  Fine-tuning Large Language Models (LLMs) on some task-specific datasets has been a primary use of LLMs.<n>This paper presents a theoretical framework for understanding the interplay between safety and capability in two primary safety-aware LLM fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-03-24T20:41:57Z)
• Combining Domain and Alignment Vectors to Achieve Better Knowledge-Safety Trade-offs in LLMs [67.27279184423723]
  We introduce an efficient merging-based alignment method called textscMergeAlign that interpolates the domain and alignment vectors, creating safer domain-specific models.<n>We apply textscMergeAlign on Llama3 variants that are experts in medicine and finance, obtaining substantial alignment improvements with minimal to no degradation on domain-specific benchmarks.
  arXiv  Detail & Related papers  (2024-11-11T09:32:20Z)
• Safety Layers in Aligned Large Language Models: The Key to LLM Security [43.805905164456846]
  Internal parameters in aligned LLMs can be vulnerable to security degradation when subjected to fine-tuning attacks.<n>Our work uncovers the mechanism behind security in aligned LLMs at the parameter level, identifying a small set of contiguous layers in the middle of the model.<n>We propose a novel fine-tuning approach, Safely Partial- Fine-Tuning (SPPFT), that fixes the gradient of the safety layers during fine-tuning to address the security degradation.
  arXiv  Detail & Related papers  (2024-08-30T04:35:59Z)
• What Makes and Breaks Safety Fine-tuning? A Mechanistic Study [64.9691741899956]
  Safety fine-tuning helps align Large Language Models (LLMs) with human preferences for their safe deployment. We design a synthetic data generation framework that captures salient aspects of an unsafe input. Using this, we investigate three well-known safety fine-tuning methods.
  arXiv  Detail & Related papers  (2024-07-14T16:12:57Z)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling [69.83976050879318]
  In large language models (LLMs), identifying sources of uncertainty is an important step toward improving reliability, trustworthiness, and interpretability. In this paper, we introduce an uncertainty decomposition framework for LLMs, called input clarification ensembling. Our approach generates a set of clarifications for the input, feeds them into an LLM, and ensembles the corresponding predictions.
  arXiv  Detail & Related papers  (2023-11-15T05:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.