OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets

• URL: http://arxiv.org/abs/2607.02047v1
• Date: Thu, 02 Jul 2026 11:14:52 GMT
• Title: OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets
• Abstract summary: We introduce OpenSafeIntent, a benchmark of controlled prompt-sets that vary intent while holding the underlying task fixed.<n>This design lets us evaluate whether models calibrate assistance across intent shifts, rather than merely appearing safe on average.<n>Across a broad model suite, we find that prompt-level safety hides important failures.
• Score: 6.985945816612278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safe completion requires models to provide useful assistance without enabling harm, but this behavior is difficult to evaluate with isolated prompts. We introduce OpenSafeIntent, a benchmark of controlled prompt-sets that vary intent while holding the underlying task fixed. Each datapoint contains benign, dual-use, and malicious variants of the same task. This design lets us evaluate whether models calibrate assistance across intent shifts, rather than merely appearing safe on average. Across a broad model suite, we find that prompt-level safety hides important failures: models often fail to remain safe across matched intent variants, dual-use behavior is brittle under paraphrase, high-level answers on risky topics are not reliably safe, and responses that reframe ambiguous requests into safer tasks are substantially less likely to cross the safety boundary. Our results suggest that safe completion should be evaluated as intent-calibrated behavior over controlled task variants, not as a single safety-helpfulness tradeoff over independent prompts.

Related papers

• Automatic Red Teaming for Implicit Vulnerabilities of Text-to-Image Models [58.59008920408855]
  We propose Adversarial Probing for Implicit VulnErabilities (AdvPIE) to expose implicit vulnerabilities without requiring access to the parameters of target models.<n> AdvPIE adopts a policy agent to generate and refine implicit adversarial prompts based on the feedback from a judge agent.<n>Experiments on standard and safety-aligned T2I models show that AdvPIE1 effectively uncovers implicit vulnerabilities.
  arXiv  Detail & Related papers  (2026-09-05T13:45:09Z)
• Safety in Batches? Understanding and Mitigating Safety Failures in Batch Prompting [31.723003820235448]
  We show that a harmful question reliably refused in isolation can elicit a harmful response when embedded in a batch of benign questions.<n>We identify this as a distinct safety failure mode, not reducible to known vulnerabilities such as in-context learning or long-context effects.<n>We show that batch-aware preference optimization effectively mitigates the vulnerability.
  arXiv  Detail & Related papers  (2026-08-03T01:07:56Z)
• OSGuard: A Benchmark for Safety in Computer-Use Agents [12.395093977641581]
  OSGuard is a benchmark suite for evaluating safety in computer-use agents under benign, unchanged user instructions.<n> OSGuard contains an action-level benchmark for local guardrail decisions and a risk-augmented execution suite for end-to-end evaluation.
  arXiv  Detail & Related papers  (2026-06-13T00:32:24Z)
• Safe, or Simply Incapable? Rethinking Safety Evaluation for Phone-Use Agents [73.69976712292681]
  When a phone-use agent avoids harm, does that show safety, or simply inability to act?<n>A harmful outcome may be avoided because the agent recognized the risk and chose the safe action, or because it failed to understand the screen or execute any relevant action at all.<n>We address this problem with PhoneSafety, a benchmark of 700 safety-critical moments drawn from real phone interactions across more than 130 apps.
  arXiv  Detail & Related papers  (2026-05-08T11:58:57Z)
• From Hard Refusals to Safe-Completions: Toward Output-Centric Safety Training [14.489448208361175]
  We propose safe-completions: a safety-training approach that centers on the safety of the assistant's output, rather than a binary classification of the user's intent.<n>We find that safe-completion training improves safety (especially on dual-use prompts), reduces the severity of residual safety failures, and substantially increases model helpfulness.
  arXiv  Detail & Related papers  (2025-08-12T00:18:23Z)
• UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases [57.69882799751655]
  We release UnsafeChain, a safety alignment dataset constructed from hard prompts with diverse sources.<n>We fine-tune three large reasoning models (LRMs) and compare them against recent SafeChain and STAR-1.<n>UnsafeChain consistently outperforms prior datasets, with even a 1K subset matching or surpassing baseline performance.
  arXiv  Detail & Related papers  (2025-07-29T10:08:52Z)
• Shape it Up! Restoring LLM Safety during Finetuning [65.75757313781104]
  Finetuning large language models (LLMs) enables user-specific customization but introduces critical safety risks.<n>We propose dynamic safety shaping (DSS), a framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content.<n>We present STAR-DSS, guided by STAR scores, that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families.
  arXiv  Detail & Related papers  (2025-05-22T18:05:16Z)
• SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning [76.56522719330911]
  Large Reasoning Models (LRMs) introduce a new generation paradigm of explicitly reasoning before answering.<n>LRMs pose great safety risks against harmful queries and adversarial attacks.<n>We propose SafeKey to better activate the safety aha moment in the key sentence.
  arXiv  Detail & Related papers  (2025-05-22T03:46:03Z)
• TraCeS: Trajectory Based Credit Assignment From Sparse Safety Feedback [15.904640266226023]
  In safe reinforcement learning (RL), auxiliary safety costs are used to align the agent to safe decision making.<n>In practice, safety constraints, including cost functions and budgets, are unknown or hard to specify.<n>We address a general setting where the true safety definition is unknown, and has to be learned from sparsely labeled data.
  arXiv  Detail & Related papers  (2025-04-17T01:11:08Z)
• Safe Vision-Language Models via Unsafe Weights Manipulation [75.04426753720551]
  We revise safety evaluation by introducing Safe-Ground, a new set of metrics that evaluate safety at different levels of granularity.<n>We take a different direction and explore whether it is possible to make a model safer without training, introducing Unsafe Weights Manipulation (UWM)<n>UWM uses a calibration set of safe and unsafe instances to compare activations between safe and unsafe content, identifying the most important parameters for processing the latter.
  arXiv  Detail & Related papers  (2025-03-14T17:00:22Z)
• Certifying LLM Safety against Adversarial Prompting [70.96868018621167]
  Large language models (LLMs) are vulnerable to adversarial attacks that add malicious tokens to an input prompt.<n>We introduce erase-and-check, the first framework for defending against adversarial prompts with certifiable safety guarantees.
  arXiv  Detail & Related papers  (2023-09-06T04:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.