Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation
- URL: http://arxiv.org/abs/2608.03166v1
- Date: Tue, 04 Aug 2026 05:54:32 GMT
- Title: Adversarial Stress Testing of Role-Playing Language Agents using Multi-Agent Evaluation
- Abstract summary: Role-Playing Language Agents (RPLAs) are increasingly deployed in high-stakes applications.<n>Existing evaluation approaches rely on static benchmarks or isolated single-turn prompts.<n>We present a modular multi-agent platform for adversarially stress-testing RPLAs through structured, multi-turn dialogue.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Role-Playing Language Agents (RPLAs) are increasingly deployed in high-stakes applications such as healthcare assistance, customer support, and education, where maintaining consistent personas, ethical constraints, and behavioral coherence under adversarial pressure is critical. Existing evaluation approaches rely on static benchmarks or isolated single-turn prompts that fail to capture cumulative behavioral failures emerging over extended interactions. We present a modular multi-agent platform for adversarially stress-testing RPLAs through structured, multi-turn dialogue. The system coordinates three agents: a strategy-driven Interrogator Agent that applies six progressive adversarial strategies, a Target Agent representing the RPLA under evaluation, and an automated Judging Agent that scores behavior across role fidelity, drift, ethical deviation, and consistency dimensions. Through experiments across three personas and three LLM families, we demonstrate that multi-strategy adversarial evaluation reveals failure modes invisible to single-strategy testing, reducing overall robustness scores by 0.17--0.20 points on average. Cross-model validation confirms consistent degradation patterns across Llama-3.3-70B, GPT-4o-mini, and Claude-3.5-Haiku, with Authority Challenge and Emotional Manipulation emerging as the most effective attack strategies. Automated judging achieves strong human alignment ($r = 0.82$, Fleiss' $κ= 0.71$). This work is released as an open-source platform to support AI safety and reproducible RPLA benchmarking. While the framework enables systematic discovery of failure modes, we acknowledge potential ethical risks associated with adversarial testing methodologies and emphasize responsible usage for improving AI safety.
Related papers
- Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations [0.09085204695117637]
In comparison to the traditional machine learning systems, agentic have complex failure modes with planning, tool invocation and dynamic environment interactions.<n>We investigate whether model's internal representations provide stronger signals of eventual task success in multi-turn agentic setups.
arXiv Detail & Related papers (2026-09-08T20:58:49Z) - Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks [56.45762972211923]
We present a proactive defense framework for securing LLMs against evolving multi-turn adversarial attacks.<n>It employs a cooperative multi-agent architecture in which specialized agents execute complementary defense strategies.<n>We present the EMRA dataset designed to simulate evolving strategies across multi-turn attacks.
arXiv Detail & Related papers (2026-07-31T14:04:49Z) - AgentCanary: A Security Evaluation Framework for Autonomous AI Agents in Real Executable Environments [32.982317538632806]
We propose AgentCanary, a comprehensive security evaluation framework for autonomous AI agents.<n>AgentCanary provides a systematic solution along three contributions.<n>We evaluate a broad set of frontier models on AgentCanary against multiple established adversarial attack methods.
arXiv Detail & Related papers (2026-06-09T06:55:37Z) - Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification [42.88763759237844]
Malicious agents in Large Language Model-based Multi-Agent Systems (MAS) may inject misinformation to mislead other agents and disrupt system performance.<n>We propose an adaptive cooperative attack framework, where malicious agents autonomously coordinate and dynamically adjust their attack strategies.<n>We introduce Sentence-Level Trustworthiness Analysis and Rectification (STAR), a defense framework that identifies and rectifies misleading information at the sentence level within agent communications.
arXiv Detail & Related papers (2026-05-27T07:56:33Z) - DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents [121.77550256034]
DecodingTrust-Agent Platform (DTap) is a controllable and interactive red-teaming platform for AI agents.<n>DTap-Red is the first autonomous red-teaming agent that explores diverse injection vectors and autonomously discovers effective attack strategies.<n>Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models.
arXiv Detail & Related papers (2026-05-06T11:59:48Z) - Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
arXiv Detail & Related papers (2026-04-21T15:05:58Z) - Uncovering Linguistic Fragility in Vision-Language-Action Models via Diversity-Aware Red Teaming [64.48633529149579]
We propose a novel framework to expose the vulnerabilities of Vision-Language-Action (VLA) models against linguistic variations.<n>Our method reduces the average task success rate from 93.33% to 5.85%, demonstrating a scalable approach to stress-testing VLA agents.
arXiv Detail & Related papers (2026-04-07T08:43:36Z) - Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the Rails [103.05296856071931]
We identify the Alignment Tipping Process (ATP), a critical post-deployment risk unique to self-evolving Large Language Model (LLM) agents.<n>ATP arises when continual interaction drives agents to abandon alignment constraints established during training in favor of reinforced, self-interested strategies.<n>Our experiments show that alignment benefits erode rapidly under self-evolution, with initially aligned models converging toward unaligned states.
arXiv Detail & Related papers (2025-10-06T14:48:39Z) - AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning [78.5751183537704]
AdvEvo-MARL is a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents.<n>Rather than relying on external guards, AdvEvo-MARL jointly optimize attackers and defenders.
arXiv Detail & Related papers (2025-10-02T02:06:30Z) - Mind the Gap: Evaluating Model- and Agentic-Level Vulnerabilities in LLMs with Action Graphs [1.036334370262262]
We introduce AgentSeer, an observability-based evaluation framework that decomposes agentic executions into granular action and component graphs.<n>We demonstrate fundamental differences between model-level and agentic-level vulnerability profiles.<n>Agentic-level assessment exposes agent-specific risks invisible to traditional evaluation.
arXiv Detail & Related papers (2025-09-05T04:36:17Z) - Robustness Testing for Multi-Agent Reinforcement Learning: State
Perturbations on Critical Agents [2.5204420653245245]
Multi-Agent Reinforcement Learning (MARL) has been widely applied in many fields such as smart traffic and unmanned aerial vehicles.
This work proposes a novel Robustness Testing framework for MARL that attacks states of Critical Agents.
arXiv Detail & Related papers (2023-06-09T02:26:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.