SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring

• URL: http://arxiv.org/abs/2607.18665v1
• Date: Tue, 21 Jul 2026 03:25:01 GMT
• Title: SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring
• Abstract summary: We introduce SciHazard, a real-world-grounded benchmark for scientific risks and a dataset evaluation framework for measuring harmfulness.<n> SciHazard contains 2400 hazardous questions and 600 oversafety questions across 12 disciplines, with both queries grounded in regulated entities and documented failure scenarios.<n>TextscDeHarm-Score improves agreement with expert annotations by 90.17% over the strongest baseline.
• Score: 45.617708056629766
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) increasingly support science, but they can also convert hazardous scientific knowledge into actionable misuse guidance. Existing benchmarks often rely on templated queries disconnected from real-world hazards, and employ LLM-as-a-Judge paradigms without domain grounding. To address this, we introduce SciHazard, a real-world-grounded benchmark for scientific risks and a dataset agnostic evaluation framework for measuring harmfulness. SciHazard contains 2400 hazardous questions and 600 oversafety questions across 12 disciplines, with both queries grounded in regulated entities and documented failure scenarios. To compute \textsc{DeHarm-Score} , we develop a decomposed evaluating procedure that combines query hazard severity, refusal behavior, and response-level risk. For non-refused responses, it further decomposes response-level harm into \textsc{Executability}, quantified via dynamic checklists with importance weighting, and \textsc{Net-new risk}, assessed through retrieval-augmented claim extraction and synthesis-barrier verification. An expert-validation study shows that \textsc{DeHarm-Score} improves agreement with expert annotations by 90.17\% over the strongest baseline. We benchmark 31 frontier LLMs and deep research agents in an extensive scientific safety evaluation. Notably, deep research agents yield 32.3\% higher mean \textsc{DeHarm-Score} than standard LLMs, exposing autonomous agents as a critical blind spot in current safety defenses. Code and dataset are available at https://anonymous.4open.science/r/DeharmScore-7B55.

Related papers

• Muse Spark Safety & Preparedness Report [106.21435337776768]
  Muse Spark is the latest large language model developed by Meta.<n>We first present evaluations for catastrophic risk domains under Meta's Advanced AI Scaling Framework.<n>We then discuss additional considerations, such as Muse Spark's broader content safety and behavioral profile.
  arXiv  Detail & Related papers  (2026-05-14T23:12:14Z)
• SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond [134.43113804188195]
  We introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts.<n>SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement.
  arXiv  Detail & Related papers  (2026-03-02T08:16:04Z)
• DeepEra: A Deep Evidence Reranking Agent for Scientific Retrieval-Augmented Generated Question Answering [28.427433335623217]
  We propose a Deep Evidence Reranking Agent (DeepEra) that integrates step-by-step reasoning.<n>This work is the first to comprehensively study and empirically validate innegligible SSLI issues in two-stage RAG frameworks.
  arXiv  Detail & Related papers  (2026-01-23T06:19:08Z)
• BadScientist: Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers? [21.78901120638025]
  We investigate whether fabrication-oriented paper generation agents can deceive multi-model LLM review systems.<n>Our generator employs presentation-manipulation strategies requiring no real experiments.<n>Despite provably sound aggregation mathematics, integrity checking systematically fails.
  arXiv  Detail & Related papers  (2025-10-20T18:37:11Z)
• RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration [81.38705556267917]
  Existing safety evaluation methods for large language models (LLMs) suffer from inherent limitations.<n>We introduce a theoretical framework that reconstructs the underlying risk concept space.<n>We propose RADAR, a multi-agent collaborative evaluation framework.
  arXiv  Detail & Related papers  (2025-09-28T09:35:32Z)
• SafeScientist: Toward Risk-Aware Scientific Discoveries by LLM Agents [11.817130554581436]
  We introduce textbfSafeScientist, an innovative AI scientist framework to enhance safety and ethical responsibility in AI-driven scientific exploration.<n>SafeScientist proactively refuses ethically inappropriate or high-risk tasks and rigorously emphasizes safety throughout the research process.<n>We propose textbfSciSafetyBench, a novel benchmark specifically designed to evaluate AI safety in scientific contexts.
  arXiv  Detail & Related papers  (2025-05-29T15:35:58Z)
• SOSBENCH: Benchmarking Safety Alignment on Scientific Knowledge [11.63268709958876]
  SOSBench is a regulation-grounded, hazard-focused benchmark for large language models.<n>It covers six high-risk scientific domains: chemistry, biology, medicine, pharmacology, physics, and psychology.<n>We evaluate frontier models within a unified evaluation framework using our SOSBench.
  arXiv  Detail & Related papers  (2025-05-27T17:47:08Z)
• Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models [29.569220030102986]
  We introduce textbfBeyond Safe Answers (BSA) bench, a novel benchmark comprising 2,000 challenging instances organized into three distinct SSA scenario types.<n> Evaluations of 19 state-of-the-art LRMs demonstrate the difficulty of this benchmark, with top-performing models achieving only 38.0% accuracy in correctly identifying risk rationales.<n>Our work provides a comprehensive assessment tool for evaluating and improving safety reasoning fidelity in LRMs, advancing the development of genuinely risk-aware and reliably safe AI systems.
  arXiv  Detail & Related papers  (2025-05-26T08:49:19Z)
• LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs [78.99703366417661]
  Large language models (LLMs) increasingly assist in tasks ranging from procedural guidance to autonomous experiment orchestration.<n>Such overreliance is particularly dangerous in high-stakes laboratory settings, where failures in hazard identification or risk assessment can result in severe accidents.<n>We propose the Laboratory Safety Benchmark (LabSafety Bench) to evaluate models on their ability to identify potential hazards, assess risks, and predict the consequences of unsafe actions in lab environments.
  arXiv  Detail & Related papers  (2024-10-18T05:21:05Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)
• OR-Bench: An Over-Refusal Benchmark for Large Language Models [65.34666117785179]
  Large Language Models (LLMs) require careful safety alignment to prevent malicious outputs.<n>This study proposes a novel method for automatically generating large-scale over-refusal datasets.<n>We introduce OR-Bench, the first large-scale over-refusal benchmark.
  arXiv  Detail & Related papers  (2024-05-31T15:44:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.