ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks

• URL: http://arxiv.org/abs/2605.25388v1
• Date: Mon, 25 May 2026 03:31:46 GMT
• Title: ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks
• Abstract summary: We introduce ViroBench, the first comprehensive and large-scale benchmark specifically designed for nucleotide foundation models (NFMs)<n>ViroBench evaluates models across two critical dimensions: biological understanding and latent biosecurity risk, covering 18 diverse scenarios within 4 task types.<n>ViroBench provides interpretable, diagnostic evaluations and a reproducible measurement framework for future research on viral nucleotide foundation models.
• Score: 86.89727311669937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nucleotide sequences constitute the fundamental genetic basis of biological systems, rendering viral genomic analysis critical for biomedical advancement. Despite progress in biological foundation models, specifically nucleotide foundation models (NFMs), the field lacks a unified standard for viral genomics to facilitate community development and enforce biosecurity constraints. To address this, we introduce ViroBench, the first comprehensive and large-scale benchmark specifically designed for NFMs in viral settings. ViroBench evaluates models across two critical dimensions: biological understanding and latent biosecurity risk, covering 18 diverse scenarios within 4 task types. Extensive evaluation of 66 NFMs across diverse architectures yields three critical conclusions. Firstly, NFMs exhibit a performance degradation in biological understanding under phylogenetic and temporal shifts, indicating weak extrapolation capabilities. Secondly, generation tasks reveal a decoupling between statistical likelihood and biological functional validity, posing latent biosecurity risks. Thirdly, controlled ablation studies reveal that taxonomic diversity in pretraining data outweighs parameter scale. Specifically, a lightweight baseline trained on diverse data achieves a 67.5% performance gain over its original model. Overall, ViroBench provides interpretable, diagnostic evaluations and a reproducible measurement framework for future research on viral nucleotide foundation models. The datasets and code are publicly available at https://github.com/QIANJINYDX/ViroBench.

Related papers

• An Early Warning of Emerging Biosecurity Risks in Frontier LLMs [62.01705276160883]
  We develop Intern-BioBreaker, a specialized bio-red-teaming model, together with an integrated computational-to-physical framework.<n>Within this framework, Intern-BioBreaker generates targeted jailbreak prompts to test whether aligned models can be induced to provide operational guidance.<n>Our evaluation reveals a concerning gap between text-level safeguards and the risks posed by capable scientific models.
  arXiv  Detail & Related papers  (2026-07-20T15:26:23Z)
• PhageBench: Can LLMs Understand Raw Bacteriophage Genomes? [2.9367295285731285]
  Bacteriophages play a critical role in regulating microbial ecosystems and in antibiotic alternatives.<n>PhageBench is the first benchmark designed to evaluate phage genome understanding by mirroring the workflow of bioinformatics experts.
  arXiv  Detail & Related papers  (2026-04-07T12:14:23Z)
• Biothreat Benchmark Generation Framework for Evaluating Frontier AI Models I: The Task-Query Architecture [0.38186458149494623]
  This paper describes the first component of a novel Biothreat Benchmark Generation (BBG) Framework.<n>The BBG approach is designed to help model developers and evaluators reliably measure and assess the biosecurity risk uplift and general harm potential of existing and future AI models.<n>As a pilot, the BBG is first being developed to address bacterial biological threats only.
  arXiv  Detail & Related papers  (2025-12-09T00:16:44Z)
• BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model [12.528834366422466]
  BioReason learns to produce logical, biologically coherent deductions.<n>It boosts KEGG-based disease pathway prediction accuracy from 86% to 98%.<n>It also improves variant effect prediction by an average of 15% over strong baselines.
  arXiv  Detail & Related papers  (2025-05-29T15:49:27Z)
• OmniGenBench: A Modular Platform for Reproducible Genomic Foundation Models Benchmarking [21.177773831820673]
  Genomic Foundation Models (GFMs) have emerged as a transformative approach to decoding the genome.<n>As GFMs scale up and reshape the landscape of AI-driven genomics, the field faces an urgent need for rigorous and reproducible evaluation.<n>We present OmniGenBench, a modular benchmarking platform designed to unify the data, model, benchmarking, and interpretability layers across GFMs.
  arXiv  Detail & Related papers  (2025-05-20T14:16:25Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Multi-Modal and Multi-Attribute Generation of Single Cells with CFGen [76.02070962797794]
  This work introduces CellFlow for Generation (CFGen), a flow-based conditional generative model that preserves the inherent discreteness of single-cell data.<n>CFGen generates whole-genome multi-modal single-cell data reliably, improving the recovery of crucial biological data characteristics.
  arXiv  Detail & Related papers  (2024-07-16T14:05:03Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Progress and Opportunities of Foundation Models in Bioinformatics [77.74411726471439]
  Foundations models (FMs) have ushered in a new era in computational biology, especially in the realm of deep learning. Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs. Review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases.
  arXiv  Detail & Related papers  (2024-02-06T02:29:17Z)
• Modelling Technical and Biological Effects in scRNA-seq data with Scalable GPLVMs [6.708052194104378]
  We extend a popular approach for probabilistic non-linear dimensionality reduction, the Gaussian process latent variable model, to scale to massive single-cell datasets. The key idea is to use an augmented kernel which preserves the factorisability of the lower bound allowing for fast variational inference.
  arXiv  Detail & Related papers  (2022-09-14T15:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.