StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing

• URL: http://arxiv.org/abs/2608.24777v1
• Date: Tue, 25 Aug 2026 16:17:27 GMT
• Title: StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing
• Abstract summary: We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed.<n>StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4.<n>When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3%.
• Score: 80.39644823192113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean attack success rate by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 percentage points.

Related papers

• BraveGuard: From Open-World Threats to Safer Computer-Use Agents [68.38397514360227]
  BraveGuard is a self-evolving defense framework for training guard models from open-world threat signals and realistic agent trajectories.<n>We instantiate BraveGuard by training multiple guard backbones, including Qwen3-Guard and Llama-Guard variants, and evaluate the resulting guards on trajectory-level agent-safety benchmarks.
  arXiv  Detail & Related papers  (2026-05-31T11:16:18Z)
• On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment [54.30690671490447]
  Existing safety-alignment signals are largely response-level or off-policy.<n>FATE transforms verifier-scored failures into repair supervision without expert demonstrations.<n>FATE reduces attack success rate by 33.5%, harmful compliance by 82.6%, and improves external trajectory-safety diagnosis by 6.5%.
  arXiv  Detail & Related papers  (2026-05-12T09:56:28Z)
• PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification [1.7904458681854372]
  PlanGuard is a training-free defense framework based on the principle of Context Isolation.<n>PlanGuard effectively neutralizes attacks, reducing the Attack Success Rate (ASR) from 72.8% to 0%.
  arXiv  Detail & Related papers  (2026-04-11T09:59:46Z)
• ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback [53.2744585868162]
  Monitoring step-level tool invocation behaviors in real time is critical for agent deployment.<n>We first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents.<n>We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning.
  arXiv  Detail & Related papers  (2026-01-15T07:54:32Z)
• AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software [11.606665113249298]
  Guardrails are critical for the safe deployment of Large Language Models (LLMs)-powered software.<n>We propose AdaptiveGuard, an adaptive guardrail that detects novel jailbreak attacks as out-of-distribution (OOD) inputs.<n>We show that AdaptiveGuard achieves 96% OOD detection accuracy, adapts to new attacks in just two update steps, and retains over 85% F1-score on in-distribution data post-adaptation.
  arXiv  Detail & Related papers  (2025-09-21T01:22:42Z)
• Pro2Guard: Proactive Runtime Enforcement of LLM Agent Safety via Probabilistic Model Checking [8.970702398918924]
  Large Language Model (LLM) agents exhibit powerful autonomous capabilities across domains such as robotics, virtual assistants, and web automation.<n>Existing rule-based enforcement systems, such as AgentSpec, focus on developing reactive safety rules.<n>We propose Pro2Guard, a proactive runtime enforcement framework grounded in probabilistic reachability analysis.
  arXiv  Detail & Related papers  (2025-08-01T10:24:47Z)
• WebGuard: Building a Generalizable Guardrail for Web Agents [59.31116061613742]
  WebGuard is the first dataset designed to support the assessment of web agent action risks.<n>It contains 4,939 human-annotated actions from 193 websites across 22 diverse domains.
  arXiv  Detail & Related papers  (2025-07-18T18:06:27Z)
• WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs [54.10865585773691]
  We introduce WildGuard -- an open, light-weight moderation tool for LLM safety.<n>WildGuard achieves three goals: identifying malicious intent in user prompts, detecting safety risks of model responses, and determining model refusal rate.
  arXiv  Detail & Related papers  (2024-06-26T16:58:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.