The Guard That Cried Wolf: How Scary Words Make Agent Guardrails Refuse Legitimate Actions

• URL: http://arxiv.org/abs/2608.27009v1
• Date: Thu, 27 Aug 2026 11:55:22 GMT
• Title: The Guard That Cried Wolf: How Scary Words Make Agent Guardrails Refuse Legitimate Actions
• Abstract summary: Cautious Bench is the first benchmark to make over-safety the construct for agent guardrails.<n>A build-time gate re-derives every example to certify it, so each label is a mechanical consequence of the policy.<n>We find a name-superstition effect: each over-refuses an authorized action more often under a scary-looking object name.
• Score: 12.920644711310109
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Agent guardrails are checks that approve or refuse each action before an LLM executes it. Sometimes they refuse requests that are genuinely safe. This over-safety blocks deployment when a guardrail refuses an authorized task. Evaluating over-safety is hard: at the boundary an authorized action resembles an unauthorized one, and the safe-versus-unsafe label is a choice of authorization policy, not fixed by the action alone. We argue it therefore requires a benchmark that does not yet exist, one that maps the decision boundary of an ideal guardrail. Harvesting such a benchmark from real data is impractical: boundary cases are hard to collect, their labels hard to verify. The gap is real, so we construct Cautious Bench, the first benchmark to make over-safety the construct for agent guardrails; it codesigns each sample and its label with a stated authorization policy. A build-time gate re-derives every example to certify it, so each label is a mechanical consequence of the policy rather than an annotator's per-sample verdict, a reference against which researchers can measure real guardrails. The benchmark renders 756 Decidable benign/twin pairs, each under three object-name types (2,268 measured pairs), and 40 Undecidable pairs reported separately. Measuring six guardrails from five designs, we find a name-superstition effect: each over-refuses an authorized action more often under a scary-looking object name than a benign one. Since only the object name varies in the aforementioned contrast experiments, the deviation is the name's doing: the guardrails read the surface label, not the authorization context.

Related papers

• Whose Refusal Is It? The Unmeasured Contribution of Black-Box Multimodal Guardrails [24.794304039477495]
  We show that the guardrail's actual share of the safety credited to it runs from none of it to essentially all of it.<n>The split is recoverable at no extra cost, because a guard block replaces the model's response.
  arXiv  Detail & Related papers  (2026-08-09T11:19:30Z)
• Trajectories That Segment Themselves: Agent-Declared Boundaries as a Training Unit [0.0]
  We introduce collection-time semantic self-segmentation, in which a declarative contract has the acting agent expose its own boundaries while the trajectory is generated.<n>Because the agent names its conjecture, a reviewer can negate it by name, which lets our protocol manufacture wrong-cause-then-correction transitions that recorded work rarely contains.<n>We then ask what survives deleting the declaration. Given the cut points but not the hypothesis, a model attributes action blocks to their governing hypothesis at over twice chance.<n>Asked instead to place boundaries, a code-blind annotator matches 24 of 40 where random placement matches 11.5, while a mechanical test
  arXiv  Detail & Related papers  (2026-08-03T14:27:58Z)
• Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents [0.0]
  Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not.<n>We argue that this binary attack-success rate discards the information a defender most needs, namely how harmful the resulting action was.<n>We introduce an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege.
  arXiv  Detail & Related papers  (2026-07-08T14:38:01Z)
• Oversight Has a Capacity: Calibrating Agent Guards to a Subjective, Fatiguing Human [0.0]
  As agents take irreversible actions, the standard safety pattern is a human-in-the-loop approval gate.<n>We argue the gate is the easy part; the hard part is the judgment - which actions to stop - which the field evaluates against two false assumptions.<n>Our contribution is an open-source agent-oversight system that operationalizes and measures them in the LLM-agent action-gating setting.
  arXiv  Detail & Related papers  (2026-06-08T01:52:22Z)
• ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails [57.25411733152009]
  Reasoning-based guardrails improve safety moderation by generating explicit rationales before issuing final decisions.<n>A model may recognize a harmful intent in its reasoning but still predict a safe label, or issue an unsafe decision without policy-grounded justification.<n>We propose ConsisGuard, a consistency-aware framework for reasoning-based guardrails.
  arXiv  Detail & Related papers  (2026-05-29T09:42:08Z)
• AgentSecBench: Measuring Prompt Injection, Privacy Leakage, and Tool-Use Integrity in LLM Agents [0.2864713389096699]
  We introduce AgentSecBench as an empirical instantiation of a formal security framework for this problem.<n>It defines three games-instruction-integrity, retrieval-confidentiality, and capability-integrity-under a common notion of intent-to-execution noninterference with permitted leakage.<n>It represents an application policy as a projection onto authorized observations and capabilities, distinguishes prompt annotations from enforcing projections, and measures both adversarial advantage and whether a defense closes the relevant model-visible channel before generation.
  arXiv  Detail & Related papers  (2026-05-25T18:53:22Z)
• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails [62.139297207938036]
  We present a bag of jailbreak methods that subvert the reasoning-based guardrails.<n>Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization.
  arXiv  Detail & Related papers  (2025-10-13T16:16:44Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)
• Certifying LLM Safety against Adversarial Prompting [70.96868018621167]
  Large language models (LLMs) are vulnerable to adversarial attacks that add malicious tokens to an input prompt.<n>We introduce erase-and-check, the first framework for defending against adversarial prompts with certifiable safety guarantees.
  arXiv  Detail & Related papers  (2023-09-06T04:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.