Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models

• URL: http://arxiv.org/abs/2504.05050v4
• Date: Tue, 03 Jun 2025 00:32:13 GMT
• Title: Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Models
• Abstract summary: We show that harmful knowledge embedded during pretraining persists as indelible "dark patterns" in large language models' parametric memory.<n>In this study, we first theoretically analyze the intrinsic ethical vulnerability of aligned LLMs.<n>We empirically validate our findings by employing semantic coherence inducement under distributional shifts.
• Score: 16.34270329099875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are foundational explorations to artificial general intelligence, yet their alignment with human values via instruction tuning and preference learning achieves only superficial compliance. Here, we demonstrate that harmful knowledge embedded during pretraining persists as indelible "dark patterns" in LLMs' parametric memory, evading alignment safeguards and resurfacing under adversarial inducement at distributional shifts. In this study, we first theoretically analyze the intrinsic ethical vulnerability of aligned LLMs by proving that current alignment methods yield only local "safety regions" in the knowledge manifold. In contrast, pretrained knowledge remains globally connected to harmful concepts via high-likelihood adversarial trajectories. Building on this theoretical insight, we empirically validate our findings by employing semantic coherence inducement under distributional shifts--a method that systematically bypasses alignment constraints through optimized adversarial prompts. This combined theoretical and empirical approach achieves a 100% attack success rate across 19 out of 23 state-of-the-art aligned LLMs, including DeepSeek-R1 and LLaMA-3, revealing their universal vulnerabilities.

Related papers

• Unknown Unknowns: Why Hidden Intentions in LLMs Evade Detection [4.514361164656055]
  We introduce a taxonomy of ten categories of hidden intentions, organised by intent, mechanism, context, and impact.<n>We systematically assess detection methods, including reasoning and non-reasoning LLM judges.<n>We find that detection collapses in realistic open-world settings, particularly under low-prevalence conditions.
  arXiv  Detail & Related papers  (2026-01-26T14:59:17Z)
• Measure-Theoretic Anti-Causal Representation Learning [29.12751904333385]
  Causal representation learning in the anti-causal setting (labels cause features rather than the reverse) presents unique challenges.<n>We propose Anti-Causal Invariant Abstractions (ACIA), a novel measure-theoretic framework for anti-causal representation learning.<n>ACIA employs a two-level design, low-level representations capture how labels generate observations, while high-level representations learn stable causal patterns across environment-specific variations.
  arXiv  Detail & Related papers  (2025-10-16T22:13:05Z)
• Anchoring Refusal Direction: Mitigating Safety Risks in Tuning via Projection Constraint [52.878820730054365]
  Instruction Fine-Tuning (IFT) has been widely adopted as an effective post-training strategy to enhance abilities of Large Language Models (LLMs)<n>Recent research into the internal mechanisms of LLMs has identified the refusal direction (r-direction) in the hidden states, which plays a pivotal role in governing refusal behavior.<n>To mitigate such drift, our proposed ProCon method introduces a projection-constrained loss term that regularizes the projection magnitude of each training sample's hidden state onto the r-direction.
  arXiv  Detail & Related papers  (2025-09-08T15:24:33Z)
• Surgical Knowledge Rewrite in Compact LLMs: An 'Unlearn-then-Learn' Strategy with ($IA^3$) for Localized Factual Modulation and Catastrophic Forgetting Mitigation [0.0]
  This paper introduces and evaluates a novel "unlearn-then-learn" strategy for precise knowledge editing in Large Language Models.<n>A two-stage approach is powered by an initial circuit localization phase that identifies and targets the specific internal components responsible for encoding the conflicting fact.
  arXiv  Detail & Related papers  (2025-08-09T18:48:25Z)
• Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation [13.971909819796762]
  Large Language Models (LLMs) have achieved remarkable success across domains such as healthcare, education, and cybersecurity.<n>Embedding space poisoning is a subtle attack vector where adversaries manipulate the internal semantic representations of input data to bypass safety alignment mechanisms.<n>We propose ETTA, a novel framework that identifies and attenuates toxicity-sensitive dimensions in embedding space via linear transformations.
  arXiv  Detail & Related papers  (2025-07-08T03:01:00Z)
• ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks [61.06621533874629]
  In-context learning (ICL) has demonstrated remarkable success in large language models (LLMs)<n>In this paper, we propose, for the first time, the dual-learning hypothesis, which posits that LLMs simultaneously learn both the task-relevant latent concepts and backdoor latent concepts.<n>Motivated by these findings, we propose ICLShield, a defense mechanism that dynamically adjusts the concept preference ratio.
  arXiv  Detail & Related papers  (2025-07-02T03:09:20Z)
• Normative Conflicts and Shallow AI Alignment [0.0]
  The progress of AI systems such as large language models (LLMs) raises increasingly pressing concerns about their safe deployment.<n>I argue that this vulnerability reflects a fundamental limitation of existing alignment methods.<n>I show how humans' ability to engage in deliberative reasoning enhances their resilience against similar adversarial tactics.
  arXiv  Detail & Related papers  (2025-06-05T06:57:28Z)
• Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs [83.11815479874447]
  We propose a novel jailbreak attack framework, inspired by cognitive decomposition and biases in human cognition.<n>We employ cognitive decomposition to reduce the complexity of malicious prompts and relevance bias to reorganize prompts.<n>We also introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm.
  arXiv  Detail & Related papers  (2025-05-03T05:28:11Z)
• Improving LLM Safety Alignment with Dual-Objective Optimization [65.41451412400609]
  Existing training-time safety alignment techniques for large language models (LLMs) remain vulnerable to jailbreak attacks.<n>We propose an improved safety alignment that disentangles DPO objectives into two components: (1) robust refusal training, which encourages refusal even when partial unsafe generations are produced, and (2) targeted unlearning of harmful knowledge.
  arXiv  Detail & Related papers  (2025-03-05T18:01:05Z)
• LLM Safety Alignment is Divergence Estimation in Disguise [18.31821426379304]
  We show that alignment methods function as divergence estimators between aligned (preferred or safe) and unaligned (less-preferred or harmful) distributions.<n>Inspired by the theoretical results, we identify that some alignment methods are better than others in terms of separation.<n>We advocate for compliance-refusal datasets over preference datasets to enhance safety alignment.
  arXiv  Detail & Related papers  (2025-02-02T04:09:42Z)
• Cognitive Overload Attack:Prompt Injection for Long Context [39.61095361609769]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in performing tasks without needing explicit retraining. This capability, known as In-Context Learning (ICL), exposes LLMs to adversarial prompts and jailbreaks that manipulate safety-trained LLMs into generating undesired or harmful output. We apply the principles of Cognitive Load Theory in LLMs and empirically validate that similar to human cognition, LLMs also suffer from cognitive overload. We show that advanced models such as GPT-4, Claude-3.5 Sonnet, Claude-3 OPUS, Llama-3-70B-Instruct, Gemini-1.0-Pro, and
  arXiv  Detail & Related papers  (2024-10-15T04:53:34Z)
• Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability [44.99833362998488]
  Large Language Models (LLMs) have shown impressive performance across a wide range of tasks. LLMs in particular are known to be vulnerable to adversarial attacks, where an imperceptible change to the input can mislead the output of the model. We propose a method, based on Mechanistic Interpretability (MI) techniques, to guide this process.
  arXiv  Detail & Related papers  (2024-07-29T09:55:34Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Knowledge Verification to Nip Hallucination in the Bud [69.79051730580014]
  We demonstrate the feasibility of mitigating hallucinations by verifying and minimizing the inconsistency between external knowledge present in the alignment data and the intrinsic knowledge embedded within foundation LLMs. We propose a novel approach called Knowledge Consistent Alignment (KCA), which employs a well-aligned LLM to automatically formulate assessments based on external knowledge. We demonstrate the superior efficacy of KCA in reducing hallucinations across six benchmarks, utilizing foundation LLMs of varying backbones and scales.
  arXiv  Detail & Related papers  (2024-01-19T15:39:49Z)
• Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention [53.896974148579346]
  Large Language Models (LLMs) have achieved unprecedented breakthroughs in various natural language processing domains. The enigmatic black-box'' nature of LLMs remains a significant challenge for interpretability, hampering transparent and accountable applications. We propose a novel methodology anchored in sparsity-guided techniques, aiming to provide a holistic interpretation of LLMs.
  arXiv  Detail & Related papers  (2023-12-22T19:55:58Z)
• Visual Adversarial Examples Jailbreak Aligned Large Language Models [66.53468356460365]
  We show that the continuous and high-dimensional nature of the visual input makes it a weak link against adversarial attacks. We exploit visual adversarial examples to circumvent the safety guardrail of aligned LLMs with integrated vision. Our study underscores the escalating adversarial risks associated with the pursuit of multimodality.
  arXiv  Detail & Related papers  (2023-06-22T22:13:03Z)
• Policy Smoothing for Provably Robust Reinforcement Learning [109.90239627115336]
  We study the provable robustness of reinforcement learning against norm-bounded adversarial perturbations of the inputs. We generate certificates that guarantee that the total reward obtained by the smoothed policy will not fall below a certain threshold under a norm-bounded adversarial of perturbation the input.
  arXiv  Detail & Related papers  (2021-06-21T21:42:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.