PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction

• URL: http://arxiv.org/abs/2605.01945v1
• Date: Sun, 03 May 2026 16:11:27 GMT
• Title: PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction
• Abstract summary: PepSpecBench is a unified benchmark for peptide MS/MS spectrum prediction.<n>It standardizes data preprocessing across complementary public datasets.<n>It also enforces a strict backbone-disjoint splitting strategy to eliminate sequence leakage.
• Score: 17.33669468355787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tandem mass spectrometry provides a high-throughput framework for identifying and quantifying proteins in complex biological samples. In computational proteomics, predicting peptide MS/MS spectra is a critical task, enabling downstream applications such as large-scale peptide identification and quantification. While deep learning architectures have substantially improved prediction accuracy, three evaluation challenges obscure the true progress of the field. First, inconsistent data preprocessing and incompatible model output spaces hinder fair model comparison. Second, flawed data splitting strategies can permit hidden sequence leakage and inflate reported performance. Third, existing evaluations typically lack comprehensive cross-species benchmarking and systematic assessment of model robustness to influential experimental conditions. To address these challenges, we propose PepSpecBench, a unified benchmark for peptide MS/MS spectrum prediction. PepSpecBench standardizes data preprocessing across complementary public datasets, enforces a strict backbone-disjoint splitting strategy to eliminate sequence leakage, and evaluates diverse architectures within a shared fragment-ion representation space. It further introduces a comprehensive multi-species evaluation suite and physically grounded metadata perturbation probes to assess model robustness and instrument awareness. We uncover previously unrecognized performance discrepancies and robustness limitations across six representative models, providing actionable insights for future model design, evaluation and practical deployment.

Related papers

• MedSegBenchmarker: A Raw-Count-First Framework for Controlled 2D Medical Image Segmentation Benchmarks [1.0230975056328007]
  We present MEDSEGBENCHMARKER (MSB), a configuration-driven framework for controlled benchmarking of 2D datasets.<n>MSB exports sample- and class-level pixel counts and predictions together with the evaluation context.<n>We demonstrate MSB in a case study involving three heterogeneous 2D datasets and multiple vision models evaluated at 256- and 512-pixel input resolutions.
  arXiv  Detail & Related papers  (2026-08-30T09:20:38Z)
• CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design [3.9585609151666934]
  Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years.<n>The field lacks a standardized benchmark for fair comparison and model development.<n>textscChimera-Bench is the largest dataset of its kind for the antibody design problem.
  arXiv  Detail & Related papers  (2026-03-13T05:06:58Z)
• Procrustean Bed for AI-Driven Retrosynthesis: A Unified Framework for Reproducible Evaluation [0.0]
  RetroCast is a unified evaluation suite that standardizes heterogeneous model outputs into a common schema.<n>We evaluate leading search-based and sequence-based algorithms on a new suite of standardized benchmarks.
  arXiv  Detail & Related papers  (2025-12-08T01:26:39Z)
• SeFEF: A Seizure Forecasting Evaluation Framework [0.0]
  We introduce a Python-based framework aimed at streamlining the development, assessment, and documentation of seizure forecasting algorithms.<n>The framework automates data labeling, cross-validation splitting, forecast post-processing, performance evaluation, and reporting.<n>It supports various forecasting horizons and includes a model card that documents implementation details, training and evaluation settings, and performance metrics.
  arXiv  Detail & Related papers  (2025-10-13T11:10:27Z)
• PRING: Rethinking Protein-Protein Interaction Prediction from Pairs to Graphs [88.98041407783502]
  PRING is the first benchmark that evaluates protein-protein interaction prediction from a graph-level perspective.<n> PRING curates a high-quality, multi-species PPI network dataset comprising 21,484 proteins and 186,818 interactions.
  arXiv  Detail & Related papers  (2025-07-07T15:21:05Z)
• DISPROTBENCH: A Disorder-Aware, Task-Rich Benchmark for Evaluating Protein Structure Prediction in Realistic Biological Contexts [76.59606029593085]
  DisProtBench is a benchmark for evaluating protein structure prediction models (PSPMs) under structural disorder and complex biological conditions.<n>DisProtBench spans three key axes: data complexity, task diversity, and Interpretability.<n>Results reveal significant variability in model robustness under disorder, with low-confidence regions linked to functional prediction failures.
  arXiv  Detail & Related papers  (2025-06-18T23:58:22Z)
• MeToken: Uniform Micro-environment Token Boosts Post-Translational Modification Prediction [65.33218256339151]
  Post-translational modifications (PTMs) profoundly expand the complexity and functionality of the proteome. Existing computational approaches predominantly focus on protein sequences to predict PTM sites, driven by the recognition of sequence-dependent motifs. We introduce the MeToken model, which tokenizes the micro-environment of each acid, integrating both sequence and structural information into unified discrete tokens.
  arXiv  Detail & Related papers  (2024-11-04T07:14:28Z)
• AsEP: Benchmarking Deep Learning Methods for Antibody-specific Epitope Prediction [12.433560411515575]
  We introduce a filtered antibody-antigen complex structure dataset, AsEP. AsEP is the largest of its kind and provides clustered groups. We propose a novel method, WALLE, which leverages both protein language models and structural modeling from graph neural networks.
  arXiv  Detail & Related papers  (2024-07-25T16:43:56Z)
• NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics [58.03989832372747]
  We present the first unified benchmark NovoBench for emphde novo peptide sequencing. It comprises diverse mass spectrum data, integrated models, and comprehensive evaluation metrics. Recent methods, including DeepNovo, PointNovo, Casanovo, InstaNovo, AdaNovo and $pi$-HelixNovo are integrated into our framework.
  arXiv  Detail & Related papers  (2024-06-16T08:23:21Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models [95.8037674226622]
  We introduce a 3-dimensional evaluation metric that characterizes the fidelity, diversity and generalization performance of any generative model in a domain-agnostic fashion. Our metric unifies statistical divergence measures with precision-recall analysis, enabling sample- and distribution-level diagnoses of model fidelity and diversity.
  arXiv  Detail & Related papers  (2021-02-17T18:25:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.