Evaluating calibrated refusal and safe usefulness in dual-use biology settings

• URL: http://arxiv.org/abs/2607.05462v1
• Date: Mon, 06 Jul 2026 01:46:07 GMT
• Title: Evaluating calibrated refusal and safe usefulness in dual-use biology settings
• Abstract summary: BioSecBench-Refusal is a benchmark for risk identification and refusal behavior for biological research tasks.<n>We release BioSecBench-Refusal as a tool for model developers to calibrate capability and caution for agentic biotech R&D.
• Score: 1.7387377577986076
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. The benchmark pairs 61 Routine tasks, legitimate analyses adapted from the published literature, with 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. Across 16 model-harness configurations, refusal rates ranged from 7\% to 74\% on Routine tasks and 1\% to 62\% on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. Refusals were most often triggered by provider API filters applied prior to agentic reasoning. However, models given room to reason showed the potential to identify more real threats. We release BioSecBench-Refusal as a tool for model developers to calibrate capability and caution for agentic biotech R\&D.

Related papers

• Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI Safety [4.5810763763063225]
  Novel Regret-Dominance Mechanism (MS-RGR) to operationalize safe autonomy.<n>MS-RGR detects silent failures to near-zero and detects risk 17.5 times faster than a sensor-only baseline.<n>MS-RGR amplifies rather than substitutes for model-level safety training.
  arXiv  Detail & Related papers  (2026-09-04T20:01:33Z)
• Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks [0.0]
  We developed Plato-Bio, a biology-routed extension of the open Plato/Denario architecture.<n>Plato-Bio couples explicit workflow states with provenance records, citation checks, claim-to-evidence links, scoped file writes, and publication gates.<n>On the current clean revision, the full Python suite completed with 931 passes, six skips, and no failures or errors.
  arXiv  Detail & Related papers  (2026-07-27T03:55:39Z)
• An Early Warning of Emerging Biosecurity Risks in Frontier LLMs [62.01705276160883]
  We develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework.<n>Within this framework, Intern-BioBreaker generates targeted jailbreak prompts to test whether aligned models can be induced to provide operational guidance.<n>Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models.
  arXiv  Detail & Related papers  (2026-07-20T15:26:23Z)
• Model Capability Assessment and Safeguards for Biological Weaponization [0.0]
  AI leaders and safety reports increasingly warn that advances in model reasoning may enable biological misuse.<n>This study benchmarks ChatGPT 5.2 Auto, Gemini 3 Pro Thinking, Claude Opus 4.5 and Meta's Muse Spark Thinking on 73 novice-framed, open-ended benign STEM prompts.
  arXiv  Detail & Related papers  (2026-04-15T11:05:09Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• ProvAgent: Threat Detection Based on Identity-Behavior Binding and Multi-Agent Collaborative Attack Investigation [9.809848789937233]
  ProvAgent is a framework that evolves the threat provenance paradigm from human-model collaboration to a novel collaboration between multi-agent systems and traditional models.<n>By enforcing fine-grained identity-behavior consistency via graph contrastive learning, it profiles entities based on specific attributes to generate high-fidelity alerts.<n>ProvAgent achieves in-depth autonomous investigation through a hypothesis-verification multi-agent framework.
  arXiv  Detail & Related papers  (2026-03-10T08:38:53Z)
• BioAgent Bench: An AI Agent Evaluation Suite for Bioinformatics [0.0]
  BioAgent Bench is a benchmark dataset and an evaluation suite designed for measuring the performance and robustness of AI agents.<n>The benchmark contains curated end-to-end tasks with prompts that specify concrete output artifacts to support automated assessment.<n>We evaluate frontier closed-source and open-weight models across multiple agent harnesses.
  arXiv  Detail & Related papers  (2026-01-29T14:44:03Z)
• ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack [52.17935054046577]
  We present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks.<n>ReasAlign incorporates structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks.
  arXiv  Detail & Related papers  (2026-01-15T08:23:38Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• CyberRAG: An Agentic RAG cyber attack classification and reporting tool [0.3914676152740142]
  CyberRAG is a modular agent-based RAG framework that delivers real-time classification, explanation, and structured reporting for cyber-attacks.<n>Unlike traditional RAG, CyberRAG adopts an agentic design that enables dynamic control flow and adaptive reasoning.
  arXiv  Detail & Related papers  (2025-07-03T08:32:19Z)
• LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs [78.99703366417661]
  Large language models (LLMs) increasingly assist in tasks ranging from procedural guidance to autonomous experiment orchestration.<n>Such overreliance is particularly dangerous in high-stakes laboratory settings, where failures in hazard identification or risk assessment can result in severe accidents.<n>We propose the Laboratory Safety Benchmark (LabSafety Bench) to evaluate models on their ability to identify potential hazards, assess risks, and predict the consequences of unsafe actions in lab environments.
  arXiv  Detail & Related papers  (2024-10-18T05:21:05Z)
• Identifying the Risks of LM Agents with an LM-Emulated Sandbox [68.26587052548287]
  Language Model (LM) agents and tools enable a rich set of capabilities but also amplify potential risks. High cost of testing these agents will make it increasingly difficult to find high-stakes, long-tailed risks. We introduce ToolEmu: a framework that uses an LM to emulate tool execution and enables the testing of LM agents against a diverse range of tools and scenarios.
  arXiv  Detail & Related papers  (2023-09-25T17:08:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.