RamanBench: A Large-Scale Benchmark for Machine Learning on Raman Spectroscopy

• URL: http://arxiv.org/abs/2605.02003v2
• Date: Wed, 06 May 2026 11:03:18 GMT
• Title: RamanBench: A Large-Scale Benchmark for Machine Learning on Raman Spectroscopy
• Abstract summary: RamanBench is the first large-scale, fully reproducible benchmark for Machine Learning on Raman spectroscopy.<n>It unifies 74 datasets across four domains, comprising 325,668 spectra and spanning classification and regression tasks.<n>We benchmark 28 models under a standardized protocol, including classical methods (e.g., PLS), Raman-specific (e.g., RamanNet), Tabular Foundation Model (TFM) (e.g., TabPFN) and time-series approaches.
• Score: 1.5981687661754302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine Learning (ML) has transformed many scientific fields, yet key applications still lack standardized benchmarks. Raman spectroscopy, a widely used technique for non-invasive molecular analysis, is one such field where progress is limited by fragmented datasets, inconsistent evaluation, and models that fail to capture the structure of spectral data. We introduce RamanBench, the first large-scale, fully reproducible benchmark for ML on Raman spectroscopy, consisting of streamlined data access, evaluation protocols and code, as well as a live leaderboard. It unifies 74 datasets (including 16 first released with this benchmark) across four domains, comprising 325,668 spectra and spanning classification and regression tasks under diverse experimental conditions. We benchmark 28 models under a standardized protocol, including classical methods (e.g., PLS), Raman-specific (e.g., RamanNet), Tabular Foundation Model (TFM) (e.g., TabPFN), and time-series approaches (e.g., ROCKET). TFM consistently outperform domain-specific and gradient boosting baselines, while time-series models remain competitive. However, no method generalizes across datasets, revealing a fundamental gap. Therefore, we invite the community to contribute new approaches to our living benchmark, with the potential to accelerate advances in critical applications such as medical diagnostics, biological research, and materials science.

Related papers

• MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification [11.133897813950789]
  We address the Molecule Retrieval task, which consists in recovering the chemical structure of a metabolite from its MS/MS spectrum given a set of candidate molecules.<n>We propose a unified framework encompassing recent approaches based on representation alignment and contrastive learning.<n>MSAlign is simple to implement, fast to train and consistently outperforms existing approaches across all benchmarks.
  arXiv  Detail & Related papers  (2026-05-19T12:19:35Z)
• SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models [63.357710580101894]
  Large language models (LLMs) are increasingly applied to scientific research, yet existing evaluations often fail to reflect the fine-grained capabilities required in practice.<n>We propose a new framework named SciCustom to address the problem.<n>It enables the custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs.
  arXiv  Detail & Related papers  (2026-05-19T04:41:43Z)
• TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
  Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
  arXiv  Detail & Related papers  (2026-01-30T17:28:56Z)
• Benchmarking Deep Learning Models for Raman Spectroscopy Across Open-Source Datasets [0.0]
  This study presents one of the first systematic benchmarks comparing three or more published Raman-specific deep learning classifiers across multiple open-source Raman datasets.<n>We report classification accuracies and macro-averaged F1 scores to provide a fair and reproducible comparison of deep learning models for Raman spectra based classification.
  arXiv  Detail & Related papers  (2026-01-22T16:54:53Z)
• Breaking the Modality Barrier: Generative Modeling for Accurate Molecule Retrieval from Mass Spectra [60.08608779794957]
  We propose GLMR, a Generative Language Model-based Retrieval framework.<n>In the pre-retrieval stage, a contrastive learning-based model identifies top candidate molecules as contextual priors for the input mass spectrum.<n>In the generative retrieval stage, these candidate molecules are integrated with the input mass spectrum to guide a generative model in producing refined molecular structures.
  arXiv  Detail & Related papers  (2025-11-09T07:25:53Z)
• An MLCommons Scientific Benchmarks Ontology [2.665757190742151]
  This paper introduces an ontology for scientific benchmarking developed through a unified, community-driven effort.<n>The effort consolidates a large set of disparate benchmarks and frameworks into a single taxonomy.<n>New benchmarks can be added through an open submission coordinated by the MLCommons Science Working Group.
  arXiv  Detail & Related papers  (2025-11-06T17:07:18Z)
• A Standardized Benchmark for Machine-Learned Molecular Dynamics using Weighted Ensemble Sampling [32.505127447635864]
  We introduce a modular benchmarking framework that systematically evaluates protein MD methods.<n>The framework includes a flexible, lightweight propagator interface that supports arbitrary simulation engines.<n>We contribute a dataset of nine diverse proteins, ranging from 10 to 224 residues, that span a variety of folding complexities.
  arXiv  Detail & Related papers  (2025-10-20T06:02:36Z)
• A Robust Support Vector Machine Approach for Raman COVID-19 Data Classification [0.7864304771129751]
  In this paper, we investigate the performance of a novel robust formulation for Support Vector Machine (SVM) in classifying COVID-19 samples obtained from Raman spectroscopy.<n>We derive robust counterpart models of deterministic formulations using bounded-by-norm uncertainty sets around each observation.<n>The effectiveness of our approach is validated on real-world COVID-19 datasets provided by Italian hospitals.
  arXiv  Detail & Related papers  (2025-01-29T14:02:45Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• RamanNet: A generalized neural network architecture for Raman Spectrum Analysis [4.670045009583903]
  Raman spectroscopy provides a vibrational profile of the molecules and can be used to identify different kind of materials. Despite the recent rise in Raman spectra data volume, there has not been any significant effort in developing generalized machine learning methods for Raman spectra analysis. We examine, experiment and evaluate existing methods and conjecture that neither current sequential models nor traditional machine learning models are satisfactorily sufficient to analyze Raman spectra.
  arXiv  Detail & Related papers  (2022-01-20T23:15:25Z)
• Benchmarking Graph Neural Networks [75.42159546060509]
  Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. For any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. GitHub repository has reached 1,800 stars and 339 forks, which demonstrates the utility of the proposed open-source framework.
  arXiv  Detail & Related papers  (2020-03-02T15:58:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.