Persona Jailbreaking in Large Language Models

• URL: http://arxiv.org/abs/2601.16466v1
• Date: Fri, 23 Jan 2026 05:51:35 GMT
• Title: Persona Jailbreaking in Large Language Models
• Authors: Jivnesh Sandhan, Fei Cheng, Tushar Sandhan, Yugo Murawaki,
• Abstract summary: Large Language Models (LLMs) are increasingly deployed in domains such as education, mental health and customer support.<n>Black-box persona manipulation remains unexplored, raising concerns for robustness in realistic interactions.<n>We introduce the task of persona editing, which adversarially steers LLM traits through user-side inputs under a black-box, inference-only setting.
• Score: 8.618075786777219
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are increasingly deployed in domains such as education, mental health and customer support, where stable and consistent personas are critical for reliability. Yet, existing studies focus on narrative or role-playing tasks and overlook how adversarial conversational history alone can reshape induced personas. Black-box persona manipulation remains unexplored, raising concerns for robustness in realistic interactions. In response, we introduce the task of persona editing, which adversarially steers LLM traits through user-side inputs under a black-box, inference-only setting. To this end, we propose PHISH (Persona Hijacking via Implicit Steering in History), the first framework to expose a new vulnerability in LLM safety that embeds semantically loaded cues into user queries to gradually induce reverse personas. We also define a metric to quantify attack success. Across 3 benchmarks and 8 LLMs, PHISH predictably shifts personas, triggers collateral changes in correlated traits, and exhibits stronger effects in multi-turn settings. In high-risk domains mental health, tutoring, and customer support, PHISH reliably manipulates personas, validated by both human and LLM-as-Judge evaluations. Importantly, PHISH causes only a small reduction in reasoning benchmark performance, leaving overall utility largely intact while still enabling significant persona manipulation. While current guardrails offer partial protection, they remain brittle under sustained attack. Our findings expose new vulnerabilities in personas and highlight the need for context-resilient persona in LLMs. Our codebase and dataset is available at: https://github.com/Jivnesh/PHISH

Related papers

• Too Good to be Bad: On the Failure of LLMs to Role-Play Villains [69.0500092126915]
  Large Language Models (LLMs) are increasingly tasked with creative generation, including the simulation of fictional characters.<n>We hypothesize that the safety alignment of modern LLMs creates a fundamental conflict with the task of authentically role-playing morally ambiguous or villainous characters.<n>We introduce the Moral RolePlay benchmark, a new dataset featuring a four-level moral alignment scale and a balanced test set for rigorous evaluation.<n>Our large-scale evaluation reveals a consistent, monotonic decline in role-playing fidelity as character morality decreases.
  arXiv  Detail & Related papers  (2025-11-07T03:50:52Z)
• Friend or Foe: How LLMs' Safety Mind Gets Fooled by Intent Shift Attack [53.34204977366491]
  Large language models (LLMs) remain vulnerable to jailbreaking attacks despite their impressive capabilities.<n>In this paper, we introduce ISA (Intent Shift Attack), which obfuscates LLMs about the intent of the attacks.<n>Our approach only needs minimal edits to the original request, and yields natural, human-readable, and seemingly harmless prompts.
  arXiv  Detail & Related papers  (2025-11-01T13:44:42Z)
• Evaluating & Reducing Deceptive Dialogue From Language Models with Multi-turn RL [64.3268313484078]
  Large Language Models (LLMs) interact with millions of people worldwide in applications such as customer support, education and healthcare.<n>Their ability to produce deceptive outputs, whether intentionally or inadvertently, poses significant safety concerns.<n>We investigate the extent to which LLMs engage in deception within dialogue, and propose the belief misalignment metric to quantify deception.
  arXiv  Detail & Related papers  (2025-10-16T05:29:36Z)
• PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions [14.497181581363288]
  PersonaFuse is a novel framework that enables Large Language Models to adapt and express different personalities.<n>Tests show PersonaFuse substantially outperforms baseline models across multiple dimensions of social-emotional intelligence.<n> PersonaFuse also delivers consistent improvements in downstream human-centered applications.
  arXiv  Detail & Related papers  (2025-09-09T03:39:28Z)
• Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts [79.1081247754018]
  Large Language Models (LLMs) are widely deployed in reasoning, planning, and decision-making tasks.<n>We propose a framework based on Contact Searching Questions(CSQ) to quantify the likelihood of deception.
  arXiv  Detail & Related papers  (2025-08-08T14:46:35Z)
• Enhancing Jailbreak Attacks on LLMs via Persona Prompts [39.73624426612256]
  Jailbreak attacks aim to exploit large language models (LLMs) by inducing them to generate harmful content, thereby revealing their vulnerabilities.<n>Previous jailbreak approaches have mainly focused on direct manipulations of harmful intent, with limited attention to the impact of persona prompts.<n>We propose a genetic algorithm-based method that automatically crafts persona prompts to bypass LLM's safety mechanisms.
  arXiv  Detail & Related papers  (2025-07-28T12:03:22Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Bullying the Machine: How Personas Increase LLM Vulnerability [3.116718677644653]
  Large Language Models (LLMs) are increasingly deployed in interactions where they are prompted to adopt personas.<n>This paper investigates whether such persona conditioning affects model safety under bullying.
  arXiv  Detail & Related papers  (2025-05-19T04:32:02Z)
• Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge [1.1666234644810893]
  Small models outperform larger ones in safety, suggesting that training and architecture may matter more than scale.<n>No model is fully robust to adversarial elicitation, with jailbreak attacks using low-resource languages or refusal suppression proving effective.
  arXiv  Detail & Related papers  (2025-04-10T16:00:59Z)
• Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control [44.326363467045496]
  Large Language Models (LLMs) have become a critical area of research in Reinforcement Learning from Human Feedback (RLHF) representation engineering offers a new, training-free approach. This technique leverages semantic features to control the representation of LLM's intermediate hidden states. It is difficult to encode various semantic contents, like honesty and safety, into a singular semantic feature.
  arXiv  Detail & Related papers  (2024-11-04T08:36:03Z)
• LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts [88.96201324719205]
  Safety concerns in large language models (LLMs) have gained significant attention due to their exposure to potentially harmful data during pre-training.<n>We identify a new safety vulnerability in LLMs, where seemingly benign prompts, semantically related to harmful content, can bypass safety mechanisms.<n>We introduce a novel attack method, textitActorBreaker, which identifies actors related to toxic prompts within pre-training distribution.
  arXiv  Detail & Related papers  (2024-10-14T16:41:49Z)
• Towards Safety and Helpfulness Balanced Responses via Controllable Large Language Models [64.5204594279587]
  A model that prioritizes safety will cause users to feel less engaged and assisted while prioritizing helpfulness will potentially cause harm. We propose to balance safety and helpfulness in diverse use cases by controlling both attributes in large language models.
  arXiv  Detail & Related papers  (2024-04-01T17:59:06Z)
• Intention Analysis Makes LLMs A Good Jailbreak Defender [79.4014719271075]
  We present a simple yet highly effective defense strategy, i.e., Intention Analysis ($mathbbIA$)<n>$mathbbIA$ works by triggering LLMs' inherent self-correct and improve ability through a two-stage process.<n>Experiments on varying jailbreak benchmarks show that $mathbbIA$ could consistently and significantly reduce the harmfulness in responses.
  arXiv  Detail & Related papers  (2024-01-12T13:15:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.