Language Models Can Understand Spectra: A Multimodal Model for Molecular Structure Elucidation

• URL: http://arxiv.org/abs/2508.08441v1
• Date: Mon, 04 Aug 2025 13:33:38 GMT
• Title: Language Models Can Understand Spectra: A Multimodal Model for Molecular Structure Elucidation
• Authors: Yunyue Su, Jiahui Chen, Zao Jiang, Zhenyi Zhong, Liang Wang, Qiang Liu,
• Abstract summary: We propose SpectraLLM, the first large language model designed to support multi-modal spectroscopic joint reasoning.<n>By integrating continuous and discrete spectroscopic modalities into a shared semantic space, SpectraLLM learns to uncover substructural patterns that are consistent and complementary across spectra.<n>We pretrain and fine-tune SpectraLLM in the domain of small molecules, and evaluate it on six standardized, publicly available chemical datasets.
• Score: 9.987376780022345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Structure elucidation is a fundamental technique for understanding the microscopic composition of matter and is widely applied across various disciplines in the natural sciences and engineering. However, existing methods often rely heavily on prior databases or known structural information, making it difficult to resolve unknown structures. In addition, complex structures typically require the joint analysis of multiple spectroscopic modalities. This process heavily depends on expert domain knowledge and is often accompanied by high costs in terms of both time and instrumentation. To address these challenges, we propose SpectraLLM, the first large language model designed to support multi-modal spectroscopic joint reasoning. SpectraLLM is capable of processing either single or multiple spectroscopic inputs and performing end-to-end structure elucidation. By integrating continuous and discrete spectroscopic modalities into a shared semantic space, SpectraLLM learns to uncover substructural patterns that are consistent and complementary across spectra, enabling precise molecular structure elucidation. We pretrain and fine-tune SpectraLLM in the domain of small molecules, and evaluate it on six standardized, publicly available chemical datasets. The model achieves state-of-the-art performance, significantly outperforming existing approaches trained on single modalities. Notably, SpectraLLM demonstrates strong robustness and generalization even for single-spectrum inference, while its multi-modal reasoning capability further improves the accuracy of structural prediction.

Related papers

• Generative structural elucidation from mass spectra as an iterative optimization problem [23.97077717251806]
  We introduce a computational workflow that poses structure elucidation from LC-MS/MS as an iterative optimization problem.<n>We demonstrate chromatography's performance on the NIST'20 and MassSpecGym datasets as both a standalone elucidation pipeline and as a complement to existing inverse models.
  arXiv  Detail & Related papers  (2026-02-07T21:34:38Z)
• How well can off-the-shelf LLMs elucidate molecular structures from mass spectra using chain-of-thought reasoning? [51.286853421822705]
  Large language models (LLMs) have shown promise for reasoning-intensive scientific tasks, but their capability for chemical interpretation is still unclear.<n>We introduce a Chain-of-Thought (CoT) prompting framework and benchmark that evaluate how LLMs reason about mass spectral data to predict molecular structures.<n>Our evaluation across metrics of SMILES validity, formula consistency, and structural similarity reveals that while LLMs can produce syntactically valid and partially plausible structures, they fail to achieve chemical accuracy or link reasoning to correct molecular predictions.
  arXiv  Detail & Related papers  (2026-01-09T20:08:42Z)
• Breaking the Modality Barrier: Generative Modeling for Accurate Molecule Retrieval from Mass Spectra [60.08608779794957]
  We propose GLMR, a Generative Language Model-based Retrieval framework.<n>In the pre-retrieval stage, a contrastive learning-based model identifies top candidate molecules as contextual priors for the input mass spectrum.<n>In the generative retrieval stage, these candidate molecules are integrated with the input mass spectrum to guide a generative model in producing refined molecular structures.
  arXiv  Detail & Related papers  (2025-11-09T07:25:53Z)
• MS-BART: Unified Modeling of Mass Spectra and Molecules for Structure Elucidation [20.973121120131875]
  Large-scale pretraining has proven effective in addressing data scarcity in other domains.<n>We propose MS-BART, a unified modeling framework that maps mass spectra and molecular structures into a shared token vocabulary.<n>Extensive evaluations demonstrate that MS-BART achieves SOTA performance across 5/12 key metrics on MassSpecGym and NPLIB1.
  arXiv  Detail & Related papers  (2025-10-23T14:45:28Z)
• DiffSpectra: Molecular Structure Elucidation from Spectra using Diffusion Models [66.41802970528133]
  Molecular structure elucidation from spectra is a foundational problem in chemistry.<n>Traditional methods rely heavily on expert interpretation and lack scalability.<n>We present DiffSpectra, a generative framework that directly infers both 2D and 3D molecular structures from multi-modal spectral data.
  arXiv  Detail & Related papers  (2025-07-09T13:57:20Z)
• Towards a Unified Textual Graph Framework for Spectral Reasoning via Physical and Chemical Information Fusion [33.53441350137292]
  We propose a novel multi-modal spectral analysis framework that integrates prior knowledge graphs with Large Language Models.<n>Our method bridges physical spectral measurements and chemical structural semantics by representing them in a unified Textual Graph format.<n>Our framework achieves consistently high performance across multiple spectral analysis tasks, including node-level, edge-level, and graph-level classification.
  arXiv  Detail & Related papers  (2025-06-21T16:58:30Z)
• Spectra-to-Structure and Structure-to-Spectra Inference Across the Periodic Table [60.78615287040791]
  XAStruct is a learning framework capable of both predicting XAS spectra from crystal structures and inferring local structural descriptors from XAS input.<n>XAStruct is trained on a large-scale dataset spanning over 70 elements across the periodic table.
  arXiv  Detail & Related papers  (2025-06-13T15:58:05Z)
• DiffMS: Diffusion Generation of Molecules Conditioned on Mass Spectra [60.39311767532607]
  We present DiffMS, a formula-restricted encoder-decoder generative network that achieves state-of-the-art performance on this task.<n>To develop a robust decoder that bridges latent embeddings and molecular structures, we pretrain the diffusion decoder with fingerprint-structure pairs.<n>Experiments on established benchmarks show that DiffMS outperforms existing models on de novo molecule generation.
  arXiv  Detail & Related papers  (2025-02-13T18:29:48Z)
• Biology Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models [51.316001071698224]
  We introduce Biology-Instructions, the first large-scale multi-omics biological sequences-related instruction-tuning dataset.<n>This dataset can bridge the gap between large language models (LLMs) and complex biological sequences-related tasks.<n>We also develop a strong baseline called ChatMultiOmics with a novel three-stage training pipeline.
  arXiv  Detail & Related papers  (2024-12-26T12:12:23Z)
• Unraveling Molecular Structure: A Multimodal Spectroscopic Dataset for Chemistry [0.1747623282473278]
  This dataset comprises simulated $1$H-NMR, $13$C-NMR, HSQC-NMR, Infrared, and Mass spectra for 790k molecules extracted from chemical reactions in patent data. We provide benchmarks for evaluating single-modality tasks such as structure elucidation, predicting the spectra for a target molecule, and functional group predictions.
  arXiv  Detail & Related papers  (2024-07-04T12:52:48Z)
• Compositional Representation of Polymorphic Crystalline Materials [56.80318252233511]
  We introduce PCRL, a novel approach that employs probabilistic modeling of composition to capture the diverse polymorphs from available structural information.<n>Extensive evaluations on sixteen datasets demonstrate the effectiveness of PCRL in learning compositional representation.
  arXiv  Detail & Related papers  (2023-11-17T20:34:28Z)
• Decoding Structure-Spectrum Relationships with Physically Organized Latent Spaces [6.36075035468233]
  A new semi-supervised machine learning method for the discovery of structure-spectrum relationships is developed and demonstrated. This method constructs a one-to-one mapping between individual structure descriptors and spectral trends. The RankAAE methodology produces a continuous and interpretable latent space, where each dimension can track an individual structure descriptor.
  arXiv  Detail & Related papers  (2023-01-11T21:30:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.