A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models

• URL: http://arxiv.org/abs/2608.02684v2
• Date: Wed, 05 Aug 2026 14:49:10 GMT
• Title: A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models
• Abstract summary: Large Language Models (LLMs) are accelerating biological research, yet this same capability poses a critical biosecurity threat.<n>Current safety evaluations operate in natural language and cannot determine whether a model-generated amino acid sequence is biological gibberish.<n>We release SPIKE-Bench and BioSafe-Guard to support more rigorous biosecurity evaluation of LLMs.
• Score: 24.33428502712115
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are accelerating biological research, yet this same capability poses a critical biosecurity threat: models that assist in protein engineering can equally be prompted to generate predicted toxin-like sequences, potentially lowering the barrier to biological misuse. Current safety evaluations, however, operate in natural language and cannot determine whether a model-generated amino acid sequence is biological gibberish or a computational risk signal. To address this evaluation blind spot, we introduce SPIKE-Bench, coupling 631 curated toxin-design prompts across seven functional categories with the SPIKE funnel, a three-stage protocol that filters output through compliance, biological plausibility, and predicted toxicity, producing stage-level diagnostics and an aggregate function-aware metric: the Functional Harmfulness Rate (FHR). An audit of 32 LLMs reveals that most models freely comply with toxin-design requests; FHR is driven primarily by biological generation capability rather than safety alignment, reaching 50.7%; and Refusal Rate fails to predict functional risk. As a first step toward mitigation, we provide BioSafe-Guard, a domain-specialized classifier that substantially reduces predicted functional risk while preserving benign utility. We release SPIKE-Bench and BioSafe-Guard at https://github.com/PKU-Alignment/SPIKE-Bench to support more rigorous biosecurity evaluation of LLMs.

Related papers

• An Early Warning of Emerging Biosecurity Risks in Frontier LLMs [62.01705276160883]
  We develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework.<n>Within this framework, Intern-BioBreaker generates targeted jailbreak prompts to test whether aligned models can be induced to provide operational guidance.<n>Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models.
  arXiv  Detail & Related papers  (2026-07-20T15:26:23Z)
• MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules [60.55214893687032]
  MolSafeEval is a benchmark dedicated to evaluating and analyzing the safety risks of molecular generation.<n>It integrates heterogeneous safety knowledge - ranging from toxicological databases to hazard rules - into a structured molecular safety knowledge graph.<n>By systematically revealing the safety vulnerabilities of current generative approaches, MolSafeEval offers a new lens for benchmarking molecular models.
  arXiv  Detail & Related papers  (2026-07-01T05:33:51Z)
• ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks [86.89727311669937]
  We introduce ViroBench, the first comprehensive and large-scale benchmark specifically designed for nucleotide foundation models (NFMs)<n>ViroBench evaluates models across two critical dimensions: biological understanding and latent biosecurity risk, covering 18 diverse scenarios within 4 task types.<n>ViroBench provides interpretable, diagnostic evaluations and a reproducible measurement framework for future research on viral nucleotide foundation models.
  arXiv  Detail & Related papers  (2026-05-25T03:31:46Z)
• BioShield: A Context-Aware Firewall for Securing Bio-LLMs [1.2694660563081597]
  Bio-LLMs can be exploited to generate harmful biological insights under the guise of legitimate research queries.<n>Existing safeguards, such as static prompt filtering, are insufficient when Bio-LLMs are embedded within dynamic biological and application-layer systems.<n>We present BioShield, a context-aware application-level firewall designed to secure Bio LLMs against dual-use attacks.
  arXiv  Detail & Related papers  (2026-03-23T22:18:54Z)
• Evaluating Proactive Risk Awareness of Large Language Models [30.312744244385822]
  We introduce a proactive risk awareness evaluation framework that measures whether large language models can anticipate potential harms and provide warnings before damage occurs.<n>We construct the Butterfly dataset to instantiate this framework in the environmental and ecological domain.
  arXiv  Detail & Related papers  (2026-02-24T15:00:00Z)
• Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective [104.09817371557476]
  Large language models (LLMs) have achieved impressive results across a range of natural language processing tasks.<n>Their potential to generate harmful content has raised serious safety concerns.<n>We introduce three novel multi-label benchmarks for toxicity detection.
  arXiv  Detail & Related papers  (2025-10-16T06:50:33Z)
• Resilient Biosecurity in the Era of AI-Enabled Bioweapons [0.0]
  Existing biosafety measures rely on sequence alignment and protein-protein interaction prediction to detect dangerous outputs.<n>We evaluate the performance of three leading PPI prediction tools: AlphaFold 3, AF3Complex, and SpatialPPIv2.<n>None of the tools successfully identify any of the four experimentally validated SARS-CoV-2 mutants with confirmed binding.
  arXiv  Detail & Related papers  (2025-08-30T18:09:04Z)
• DISPROTBENCH: A Disorder-Aware, Task-Rich Benchmark for Evaluating Protein Structure Prediction in Realistic Biological Contexts [76.59606029593085]
  DisProtBench is a benchmark for evaluating protein structure prediction models (PSPMs) under structural disorder and complex biological conditions.<n>DisProtBench spans three key axes: data complexity, task diversity, and Interpretability.<n>Results reveal significant variability in model robustness under disorder, with low-confidence regions linked to functional prediction failures.
  arXiv  Detail & Related papers  (2025-06-18T23:58:22Z)
• Can Large Language Models Design Biological Weapons? Evaluating Moremi Bio [0.0]
  Our study generated 1020 novel toxic proteins and 5,000 toxic small molecules.<n>In-depth computational toxicity assessments revealed that all the proteins scored high in toxicity.<n>Our findings underscore the critical need for robust governance and technical safeguards to balance rapid biotechnological innovation with biosecurity imperatives.
  arXiv  Detail & Related papers  (2025-05-22T11:27:50Z)
• SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models [50.34706204154244]
  Acquiring reasoning capabilities catastrophically degrades inherited safety alignment.<n>Certain scenarios suffer 25 times higher attack rates.<n>Despite tight reasoning-answer safety coupling, MLRMs demonstrate nascent self-correction.
  arXiv  Detail & Related papers  (2025-04-09T06:53:23Z)
• LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs [78.99703366417661]
  Large language models (LLMs) increasingly assist in tasks ranging from procedural guidance to autonomous experiment orchestration.<n>Such overreliance is particularly dangerous in high-stakes laboratory settings, where failures in hazard identification or risk assessment can result in severe accidents.<n>We propose the Laboratory Safety Benchmark (LabSafety Bench) to evaluate models on their ability to identify potential hazards, assess risks, and predict the consequences of unsafe actions in lab environments.
  arXiv  Detail & Related papers  (2024-10-18T05:21:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.