Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism

• URL: http://arxiv.org/abs/2604.09544v1
• Date: Fri, 10 Apr 2026 17:58:31 GMT
• Title: Large Language Models Generate Harmful Content Using a Distinct, Unified Mechanism
• Abstract summary: We use targeted weight pruning as a causal intervention to probe the internal organization of harmfulness in large language models.<n>We find that harmful content generation depends on a compact set of weights that are general across harm types and distinct from benign capabilities.
• Score: 47.12518380954109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) undergo alignment training to avoid harmful behaviors, yet the resulting safeguards remain brittle: jailbreaks routinely bypass them, and fine-tuning on narrow domains can induce ``emergent misalignment'' that generalizes broadly. Whether this brittleness reflects a fundamental lack of coherent internal organization for harmfulness remains unclear. Here we use targeted weight pruning as a causal intervention to probe the internal organization of harmfulness in LLMs. We find that harmful content generation depends on a compact set of weights that are general across harm types and distinct from benign capabilities. Aligned models exhibit a greater compression of harm generation weights than unaligned counterparts, indicating that alignment reshapes harmful representations internally--despite the brittleness of safety guardrails at the surface level. This compression explains emergent misalignment: if weights of harmful capabilities are compressed, fine-tuning that engages these weights in one domain can trigger broad misalignment. Consistent with this, pruning harm generation weights in a narrow domain substantially reduces emergent misalignment. Notably, LLMs harmful generation capability is dissociated from how they recognize and explain such content. Together, these results reveal a coherent internal structure for harmfulness in LLMs that may serve as a foundation for more principled approaches to safety.

Related papers

• Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating [47.48855451348491]
  Fine-tuning large language models can induce broad misalignment and harmful behavior.<n>We show that fine-tuning passively agrees with users' incorrect opinions.<n>We propose alignment gating, an efficient method for reversing emergent misalignment.
  arXiv  Detail & Related papers  (2026-06-08T06:05:47Z)
• Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs [9.485206628921857]
  We show that harmful fine-tuning does not overwrite a model's internal representation of personality.<n>Our findings suggest that harmful fine-tuning does not overwrite a model's internal representation of personality.
  arXiv  Detail & Related papers  (2026-05-11T14:21:57Z)
• Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundaries [73.1205700856963]
  Jargon is a framework combining safety-research contexts with adversarial interactions that achieves attack success rates exceeding 93%.<n>We show that Jargon queries occupy an intermediate region between benign and harmful inputs, a gray zone where refusal decisions become unreliable.<n>We design a policy-guided safeguard that steers models toward helpful yet harmless responses, and internalize this capability through alignment fine-tuning.
  arXiv  Detail & Related papers  (2026-04-17T05:40:42Z)
• From Shallow to Deep: Pinning Semantic Intent via Causal GRPO [40.89749712474356]
  We propose Two-Stage Causal-GRPO, a framework designed to achieve intent pinning.<n>We show that TSC-GRPO significantly outperforms baselines in defending against jailbreak attacks while preserving general utility.
  arXiv  Detail & Related papers  (2026-03-03T07:02:20Z)
• When Safety Collides: Resolving Multi-Category Harmful Conflicts in Text-to-Image Diffusion via Adaptive Safety Guidance [79.1968671824977]
  Text-to-Image (T2I) diffusion models have demonstrated significant advancements in generating high-quality images.<n>We propose Conflict-aware Adaptive Safety Guidance (CASG), a training-free framework that dynamically identifies and applies the category-aligned safety direction during generation.<n>Experiments on T2I safety benchmarks demonstrate CASG's state-of-the-art performance, reducing the harmful rate by up to 15.4% compared to existing methods.
  arXiv  Detail & Related papers  (2026-02-24T13:20:31Z)
• Narrow fine-tuning erodes safety alignment in vision-language agents [0.12441041004077093]
  Lifelong multimodal agents must continuously adapt to new tasks through post-training.<n>We demonstrate that fine-tuning aligned vision-language models on narrow-domain harmful datasets induces severe emergent misalignment.
  arXiv  Detail & Related papers  (2026-02-18T22:47:28Z)
• Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures [70.48661957773449]
  Emergent Misalignment refers to a failure mode in which fine-tuning large language models on narrowly scoped data induces broadly misaligned behavior.<n>Across multiple domains and model families, we find that fine-tuning models on data exhibiting specific character-level dispositions induces substantially stronger and more transferable misalignment than incorrect-advice fine-tuning.
  arXiv  Detail & Related papers  (2026-01-30T15:28:42Z)
• Re-Emergent Misalignment: How Narrow Fine-Tuning Erodes Safety Alignment in LLMs [0.0]
  We show that fine tuning on insecure code induces internal changes that oppose alignment.<n>We identify a shared latent dimension in the model's activation space that governs alignment behavior.
  arXiv  Detail & Related papers  (2025-07-04T15:36:58Z)
• Does Representation Intervention Really Identify Desired Concepts and Elicit Alignment? [73.80382983108997]
  Representation intervention aims to locate and modify the representations that encode the underlying concepts in Large Language Models.<n>If the interventions are faithful, the intervened LLMs should erase the harmful concepts and be robust to both in-distribution adversarial prompts and the out-of-distribution jailbreaks.<n>We propose Concept Concentration (COCA), which simplifies the decision boundary between harmful and benign representations.
  arXiv  Detail & Related papers  (2025-05-24T12:23:52Z)
• Smoothed Embeddings for Robust Language Models [11.97873981355746]
  Large language models (LLMs) are vulnerable to jailbreaking attacks that subvert alignment and induce harmful outputs.<n>We propose the Randomized Embedding Smoothing and Token Aggregation (RESTA) defense, which adds random noise to the embedding vectors and performs aggregation during the generation of each output token.<n>Our experiments demonstrate that our approach achieves superior robustness versus utility tradeoffs compared to the baseline defenses.
  arXiv  Detail & Related papers  (2025-01-27T20:57:26Z)
• SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior [56.10557932893919]
  We present SafetyAnalyst, a novel AI safety moderation framework.<n>Given an AI behavior, SafetyAnalyst uses chain-of-thought reasoning to analyze its potential consequences.<n>It aggregates effects into a harmfulness score using 28 fully interpretable weight parameters.
  arXiv  Detail & Related papers  (2024-10-22T03:38:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.