ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents

• URL: http://arxiv.org/abs/2608.11878v1
• Date: Wed, 12 Aug 2026 10:05:09 GMT
• Title: ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents
• Abstract summary: Large language model (LLM) agents integrated with external tools are vulnerable to prompt injections embedded in environmental states.<n>ToolHazard is a scalable environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute.
• Score: 35.530237857517626
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language model (LLM) agents integrated with external tools are vulnerable to indirect prompt injections embedded in environmental states. However, existing studies largely rely on manually implemented or reused environments, stochastic LLM-based tool simulation, and predefined injection locations, limiting scalable security research across broader domains. To bridge this gap, we propose **ToolHazard**, a scalable adversarial environment synthesis framework that reduces human engineering and supports expansion with additional seed domains and compute. Through an Environment Simulator, an Attacker Agent, and a User Simulator, ToolHazard synthesizes executable stateful environments, discovers viable injection points and generates environment-specific payloads, and constructs state-grounded long-horizon tasks. Based on ToolHazard, we build **ToolHazard-Bench** for stress-testing agents under complex workflows and diverse environmental attacks. Experiments reveal substantial agent vulnerabilities and show that injection timing and placement affect attack effectiveness. Moreover, ToolHazard-generated alignment data improves security on both ToolHazard-Bench and AgentDojo while preserving benign task utility.

Related papers

• SPADE: Self-Play in Adaptive Synthetic Executable Environments [110.80471334001574]
  We introduce SPADE, a self-play RL framework in which a single LLM plays two roles.<n>The Environment Designer writes complete, long-horizon training environments as executable code with an OpenAI Gym-style reset()/step() interface.<n>We find several components critical to success: grounding the Environment Designer on documents sampled from a large pretraining corpus, and giving it an accumulated environment memory.
  arXiv  Detail & Related papers  (2026-08-19T17:58:56Z)
• Hybrid Analysis for Secure MCP Tool Use in LLM Agents [77.68135853475535]
  We propose a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in large language model (LLM) agents.<n> MTGuard effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
  arXiv  Detail & Related papers  (2026-07-28T05:17:34Z)
• AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments [84.8170304874972]
  We present AgentGym2, a new evaluation framework with task instances grounded in real-world working demands.<n>It measures agents' ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information.
  arXiv  Detail & Related papers  (2026-07-06T14:56:51Z)
• Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use [14.813603277906623]
  We formalize OpenAgent, a problem setting characterized by distributional shifts across query, action, observation, and domain dimensions.<n>Our analysis yields a series of key insights, demonstrating that agents trained via both Supervised Fine-Tuning(SFT) and Reinforcement Learning suffer from varying degrees of performance degradation.<n>We propose Perturbation-Augmented Fine-Tuning, a disturbance-based intervention strategy for SFT that lays the foundation for enhancing agent robustness and utility in realistic environments.
  arXiv  Detail & Related papers  (2026-07-01T15:40:25Z)
• Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning [62.499592503950026]
  Large language model (LLM) have empowered autonomous agents to perform complex tasks that require multi-turn interactions with tools and environments.<n>We propose Agent World Model (AWM), a fully synthetic environment generation pipeline.<n>We scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets.
  arXiv  Detail & Related papers  (2026-02-10T18:55:41Z)
• ToolGym: an Open-world Tool-using Environment for Scalable Agent Testing and Data Curation [42.479399507055454]
  We introduce an open-world tool-using environment, built on 5,571 format unified tools across 204 commonly used apps.<n>It includes a task creation engine that synthesizes longhorizon, multi-tool with wild constraints, and a state controller that injects interruptions and failures to stress-test robustness.<n> Comprehensive evaluation of state-of-the-art LLMs reveals the misalignment between tool planning and execution abilities, the constraint following weakness of existing LLMs, and DeepSeek-v3.2's strongest robustness.
  arXiv  Detail & Related papers  (2026-01-09T21:59:31Z)
• AutoForge: Automated Environment Synthesis for Agentic Reinforcement Learning [71.4322853508083]
  Conducting reinforcement learning in simulated environments offers a cost-effective and highly scalable way to enhance language-based agents.<n>Previous work has been limited to semi-automated environment synthesis or tasks lacking sufficient difficulty, offering little breadth or depth.<n>We propose a unified pipeline for automated and scalable synthesis of simulated environments associated with high-difficulty but easily verifiable tasks.
  arXiv  Detail & Related papers  (2025-12-28T09:43:11Z)
• AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions [64.85086226439954]
  We present SAFE, a benchmark for assessing the safety of embodied VLM agents on hazardous instructions.<n> SAFE comprises three components: SAFE-THOR, SAFE-VERSE, and SAFE-DIAGNOSE.<n>We uncover systematic failures in translating hazard recognition into safe planning and execution.
  arXiv  Detail & Related papers  (2025-06-17T16:37:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.