PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI

• URL: http://arxiv.org/abs/2605.05682v2
• Date: Sat, 09 May 2026 06:52:17 GMT
• Title: PersonaTeaming: Supporting Persona-Driven Red-Teaming for Generative AI
• Abstract summary: We develop PersonaTeaming, which incorporates personas into the adversarial prompt generation process.<n>We then instantiate PersonaTeaming as PersonaTeaming Playground, a user-facing interface that enables red-teamers to author their own personas and collaborate with AI to mutate and refine prompts.<n>In a user study with 11 industry practitioners, we found that PersonaTeaming Playground enabled diverse red-teaming strategies and outputs that practitioners perceived as useful.
• Score: 28.268811472721996
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent developments in AI safety research have called for red-teaming methods that effectively surface potential risks posed by generative AI models, with growing emphasis on how red-teamers' backgrounds and perspectives shape their strategies and the risks they uncover. While automated red-teaming approaches promise to complement human red-teaming through larger-scale exploration, existing automated approaches do not account for human identities and rarely incorporate human inputs. In this work, we explore persona-driven red-teaming to advance both automated red-teaming and human-AI collaboration. We first develop PersonaTeaming Workflow, which incorporates personas into the adversarial prompt generation process to explore a wider spectrum of adversarial strategies. Compared to RainbowPlus, a state-of-the-art automated red-teaming method, PersonaTeaming Workflow achieves higher attack success rates while maintaining prompt diversity. However, since automated personas only approximate real human perspectives, we further instantiate PersonaTeaming Workflow as PersonaTeaming Playground, a user-facing interface that enables red-teamers to author their own personas and collaborate with AI to mutate and refine prompts. In a user study with 11 industry practitioners, we found that PersonaTeaming Playground enabled diverse red-teaming strategies and outputs that practitioners perceived as useful, and that AI-generated suggestions in the PersonaTeaming Playground encouraged out-of-the-box thinking even when practitioners did not follow them strictly. Together, our work advances both automated and human-in-the-loop approaches to red-teaming, while shedding light on interaction patterns and design insights for supporting human-AI collaboration in generative AI red-teaming.

Related papers

• GPT-Red: Automated Red Teaming via Self-Play at Scale [58.27696932324196]
  We introduce textbfGPT-Red, an automated red-teaming agent that is trained to discover novel prompt injection attacks against frontier LLMs.<n> GPT-Red excels at red-teaming: it reliably breaks our past models up to GPT-5.5, it finds more successful attacks than human red-teamers, and it generalizes to held-out environments.
  arXiv  Detail & Related papers  (2026-07-28T16:03:39Z)
• PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teaming [22.601361185100235]
  PersonaTeaming is an initial step towards incorporating people's background and identities in automated red-teaming.<n>We develop a dynamic persona-generating algorithm that automatically generates various persona types adaptive to different seed prompts.<n>Experiments show promising improvements (up to 144.1%) in the attack success rates of adversarial prompts through persona mutation.
  arXiv  Detail & Related papers  (2025-09-03T21:20:38Z)
• Red Teaming AI Red Teaming [11.08584572589146]
  We argue that there exists a significant gap between red teaming's original intent and its narrow focus on discovering model-level flaws in the context of generative AI.<n>We propose a comprehensive framework operationalizing red teaming in AI systems at two levels: macro-level system red teaming and micro-level model red teaming.
  arXiv  Detail & Related papers  (2025-07-07T23:23:40Z)
• Effective Automation to Support the Human Infrastructure in AI Red Teaming [5.463538170874778]
  We argue for a balanced approach that combines human expertise with automated tools to strengthen AI risk assessment.<n>We highlight key challenges in scaling automated red teaming, including considerations around worker proficiency, agency, and context-awareness.
  arXiv  Detail & Related papers  (2025-03-28T03:36:15Z)
• AI red-teaming is a sociotechnical challenge: on values, labor, and harms [3.0001147629373195]
  "Red-teaming" has quickly become the primary approach to test AI models.<n>We highlight the importance of understanding the values and assumptions behind red-teaming.
  arXiv  Detail & Related papers  (2024-12-12T22:48:19Z)
• Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI [52.138044013005]
  generative AI, particularly large language models (LLMs), become increasingly integrated into production applications. New attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems.
  arXiv  Detail & Related papers  (2024-09-23T10:18:10Z)
• A Dialogue Game for Eliciting Balanced Collaboration [64.61707514432533]
  We present a two-player 2D object placement game in which the players must negotiate the goal state themselves. We show empirically that human players exhibit a variety of role distributions, and that balanced collaboration improves task performance.
  arXiv  Detail & Related papers  (2024-06-12T13:35:10Z)
• From Persona to Personalization: A Survey on Role-Playing Language Agents [52.783043059715546]
  Recent advancements in large language models (LLMs) have boosted the rise of Role-Playing Language Agents (RPLAs) RPLAs achieve a remarkable sense of human likeness and vivid role-playing performance. They have catalyzed numerous AI applications, such as emotional companions, interactive video games, personalized assistants and copilots.
  arXiv  Detail & Related papers  (2024-04-28T15:56:41Z)
• Red-Teaming for Generative AI: Silver Bullet or Security Theater? [42.35800543892003]
  We argue that while red-teaming may be a valuable big-tent idea for characterizing GenAI harm mitigations, industry may effectively apply red-teaming and other strategies behind closed doors to safeguard AI. To move toward a more robust toolbox of evaluations for generative AI, we synthesize our recommendations into a question bank meant to guide and scaffold future AI red-teaming practices.
  arXiv  Detail & Related papers  (2024-01-29T05:46:14Z)
• The Promise and Peril of Artificial Intelligence -- Violet Teaming Offers a Balanced Path Forward [56.16884466478886]
  This paper reviews emerging issues with opaque and uncontrollable AI systems. It proposes an integrative framework called violet teaming to develop reliable and responsible AI. It emerged from AI safety research to manage risks proactively by design.
  arXiv  Detail & Related papers  (2023-08-28T02:10:38Z)
• Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks [83.37025218216888]
  We propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations. The robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications. Results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.
  arXiv  Detail & Related papers  (2020-07-24T23:35:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.