FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment

• URL: http://arxiv.org/abs/2604.04992v1
• Date: Sun, 05 Apr 2026 13:37:52 GMT
• Title: FreakOut-LLM: The Effect of Emotional Stimuli on Safety Alignment
• Abstract summary: Safety-aligned LLMs go through refusal training to reject harmful requests, but whether these mechanisms remain effective under emotionally charged stimuli is unexplored.<n>We introduce FreakOut-LLM, a framework investigating whether emotional context compromises safety alignment in adversarial settings.
• Score: 13.02804082409836
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Safety-aligned LLMs go through refusal training to reject harmful requests, but whether these mechanisms remain effective under emotionally charged stimuli is unexplored. We introduce FreakOut-LLM, a framework investigating whether emotional context compromises safety alignment in adversarial settings. Using validated psychological stimuli, we evaluate how emotional priming through system prompts affects jailbreak susceptibility across ten LLMs. We test three conditions (stress, relaxation, neutral) using scenarios from established psychological protocols, plus a no-prompt baseline, and evaluate attack success using HarmBench on AdvBench prompts. Stress priming increases jailbreak success by 65.2\% compared to neutral conditions (z = 5.93, p < 0.001; OR = 1.67, Cohen's d = 0.28), while relaxation priming produces no effect (p = 0.84). Five of ten models show significant vulnerability, with the largest effects concentrated in open-weight models. Logistic regression on 59,800 queries confirms stress as the sole significant condition predictor after controlling for prompt length (p = 0.61) and model identity. Measured psychological state strongly predicts attack success (|r|\geq0.70 across five instruments; all p < 0.001 in individual-level logistic regression). These results establish emotional context as a measurable attack surface with implications for real-world AI deployment in high-stress domains.

Related papers

• Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats [2.9434672062822376]
  Large language models (LLMs) are increasingly deployed in interactive applications.<n>They remain vulnerable to adversarial interactions that induce harmful, deceptive, or policy-violating outputs.<n>We present a verification-centric defense framework that jointly evaluates prompt intent and response harm.
  arXiv  Detail & Related papers  (2026-06-24T21:00:39Z)
• Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior [58.538864650160406]
  We run experiments across four behavioral tasks and 11 frontier LLMs.<n>We find that SR-behavior coherence exists but is selective.<n>These findings suggest that coarse personality frameworks may not be the best tools for testing deployment behavior.
  arXiv  Detail & Related papers  (2026-06-10T22:28:53Z)
• Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics [50.36375380196006]
  Jailbreak prompts can bypass alignment guardrails in large language models.<n>Prior detection approaches largely rely on a fixed metric space.<n>We show this assumption breaks under (i) pseudo-malicious prompts that are benign by intent but contain safety-related keywords.<n>We present Manifold Trajectory Kinetics (MTK), which treats an LLM as a kinetic system transforming inputs into outputs.
  arXiv  Detail & Related papers  (2026-06-05T14:49:26Z)
• Plant, Persist, Trigger: Sleeper Attack on Large Language Model Agents [64.80318459073526]
  We show that adversarial content can persist across interactions served by the same agent, making such threats harder to detect and mitigate.<n>Specifically, adversarial content may persist in the agent state, remain dormant across interactions, and later be activated by a benign user query.<n>We formalize this type of safety threat as Sleeper Attack. To evaluate it, we construct a benchmark with 1,896 instances covering six real-world harmful outcomes, three attack strategies, and three agent state targets.
  arXiv  Detail & Related papers  (2026-05-27T09:25:37Z)
• Can You Break RLVER? Probing Adversarial Robustness of RL-Trained Empathetic Agents [0.0]
  RLVER has produced language models with strong empathetic performance, evaluated on benchmarks that assume cooperative, honest users.<n>We construct the Adversarial Empathy Benchmark AEB and introduce the Emotional Consistency Score ECS to evaluate empathetic robustness under adversarial conditions.<n>We interpret the ECS--FS (Final Score) gap as a behavioral/legibility dissociation inside this simulator family, not as evidence about internal understanding or clinical readiness.
  arXiv  Detail & Related papers  (2026-05-08T02:14:12Z)
• ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models [60.14219417402433]
  Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.<n>We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three constructs drawn from standard cognitive-science accounts of non-declarative memory.<n>Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring.
  arXiv  Detail & Related papers  (2026-04-09T10:26:32Z)
• Breaking Minds, Breaking Systems: Jailbreaking Large Language Models via Human-like Psychological Manipulation [6.67891820536196]
  Psychological Jailbreak is an attack paradigm that exposes a stateful psychological attack surface in large language models.<n>Human-like Psychological Manipulation (HPM) profiles a target model's latent psychological vulnerabilities and synthesizes tailored multi-turn attack strategies.<n>HPM achieves a mean Attack Success Rate (ASR) of 88.1%, outperforming state-of-the-art attack baselines.
  arXiv  Detail & Related papers  (2025-12-20T07:02:00Z)
• Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments [4.547649832854566]
  Large language models (LLMs) increasingly operate in multi-agent and safety-critical settings, raising open questions about how their vulnerabilities scale when models interact adversarially.<n>This study examines whether larger models can systematically mitigate smaller ones - eliciting harmful restricted behavior despite alignment safeguards.
  arXiv  Detail & Related papers  (2025-11-16T15:16:33Z)
• Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines [5.249698789320767]
  PsyCrisisBench is a benchmark of 540 annotated transcripts from the Hangzhou Psychological Assistance Hotline.<n>Assessing four tasks: mood status recognition, suicidal ideation detection, suicide plan identification, and risk assessment.<n>Open-source models like QwQ-32B performed comparably to closed-source on most tasks, though closed models retained an edge in mood detection.
  arXiv  Detail & Related papers  (2025-06-02T05:18:24Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models [57.5404308854535]
  Safety backdoor attacks in large language models (LLMs) enable the stealthy triggering of unsafe behaviors while evading detection during normal interactions. We present BEEAR, a mitigation approach leveraging the insight that backdoor triggers induce relatively uniform drifts in the model's embedding space. Our bi-level optimization method identifies universal embedding perturbations that elicit unwanted behaviors and adjusts the model parameters to reinforce safe behaviors against these perturbations.
  arXiv  Detail & Related papers  (2024-06-24T19:29:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.