Scientific Language Modeling: A Quantitative Review of Large Language Models in Molecular Science

• URL: http://arxiv.org/abs/2402.04119v1
• Date: Tue, 6 Feb 2024 16:12:36 GMT
• Title: Scientific Language Modeling: A Quantitative Review of Large Language Models in Molecular Science
• Authors: Pengfei Liu, Jun Tao, Zhixiang Ren
• Abstract summary: Large language models (LLMs) offer a fresh approach to tackle scientific problems from a natural language processing (NLP) perspective. We propose a multi-modal benchmark, named ChEBI-20-MM, and perform 1263 experiments to assess the model's compatibility with data modalities and knowledge acquisition. Our pioneering analysis offers an exploration of the learning mechanism and paves the way for advancing SLM in molecular science.
• Score: 27.874571056109758
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient molecular modeling and design are crucial for the discovery and exploration of novel molecules, and the incorporation of deep learning methods has revolutionized this field. In particular, large language models (LLMs) offer a fresh approach to tackle scientific problems from a natural language processing (NLP) perspective, introducing a research paradigm called scientific language modeling (SLM). However, two key issues remain: how to quantify the match between model and data modalities and how to identify the knowledge-learning preferences of models. To address these challenges, we propose a multi-modal benchmark, named ChEBI-20-MM, and perform 1263 experiments to assess the model's compatibility with data modalities and knowledge acquisition. Through the modal transition probability matrix, we provide insights into the most suitable modalities for tasks. Furthermore, we introduce a statistically interpretable approach to discover context-specific knowledge mapping by localized feature filtering. Our pioneering analysis offers an exploration of the learning mechanism and paves the way for advancing SLM in molecular science.

Related papers

• Molecular Graph Representation Learning Integrating Large Language Models with Domain-specific Small Models [12.744381867301353]
  We propose a novel Molecular Graph representation learning framework that integrates Large language models and Domain-specific small models. We employ a multi-modal alignment method to coordinate various modalities, including molecular graphs and their corresponding descriptive texts, to guide the pre-training of molecular representations.
  arXiv  Detail & Related papers  (2024-08-19T16:11:59Z)
• Instruction Multi-Constraint Molecular Generation Using a Teacher-Student Large Language Model [49.64512917330373]
  We introduce a multi-constraint molecular generation large language model, TSMMG, akin to a student. To train TSMMG, we construct a large set of text-molecule pairs by extracting molecular knowledge from these 'teachers' We experimentally show that TSMMG remarkably performs in generating molecules meeting complex, natural language-described property requirements.
  arXiv  Detail & Related papers  (2024-03-20T02:15:55Z)
• Bridging Text and Molecule: A Survey on Multimodal Frameworks for Molecule [16.641797535842752]
  In this paper, we present the first systematic survey on multimodal frameworks for molecules research. We begin with the development of molecular deep learning and point out the necessity to involve textual modality. Furthermore, we delves into the utilization of large language models and prompting techniques for molecular tasks and present significant applications in drug discovery.
  arXiv  Detail & Related papers  (2024-03-07T03:03:13Z)
• MolTC: Towards Molecular Relational Modeling In Language Models [28.960416816491392]
  We propose a novel framework for Molecular inTeraction prediction following Chain-of-Thought (CoT) theory termed MolTC. Our experiments, conducted across various datasets involving over 4,000,000 molecular pairs, exhibit the superiority of our method over current GNN and LLM-based baselines.
  arXiv  Detail & Related papers  (2024-02-06T07:51:56Z)
• Empowering Molecule Discovery for Molecule-Caption Translation with Large Language Models: A ChatGPT Perspective [53.300288393173204]
  Large Language Models (LLMs) have shown remarkable performance in various cross-modal tasks. In this work, we propose an In-context Few-Shot Molecule Learning paradigm for molecule-caption translation. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation.
  arXiv  Detail & Related papers  (2023-06-11T08:16:25Z)
• Scaling Vision-Language Models with Sparse Mixture of Experts [128.0882767889029]
  We show that mixture-of-experts (MoE) techniques can achieve state-of-the-art performance on a range of benchmarks over dense models of equivalent computational cost. Our research offers valuable insights into stabilizing the training of MoE models, understanding the impact of MoE on model interpretability, and balancing the trade-offs between compute performance when scaling vision-language models.
  arXiv  Detail & Related papers  (2023-03-13T16:00:31Z)
• A Molecular Multimodal Foundation Model Associating Molecule Graphs with Natural Language [63.60376252491507]
  We propose a molecular multimodal foundation model which is pretrained from molecular graphs and their semantically related textual data. We believe that our model would have a broad impact on AI-empowered fields across disciplines such as biology, chemistry, materials, environment, and medicine.
  arXiv  Detail & Related papers  (2022-09-12T00:56:57Z)
• Less is More: A Call to Focus on Simpler Models in Genetic Programming for Interpretable Machine Learning [1.0323063834827415]
  Interpretability can be critical for the safe and responsible use of machine learning models in high-stakes applications. We argue that research in GP for IML needs to focus on searching in the space of low-complexity models.
  arXiv  Detail & Related papers  (2022-04-05T08:28:07Z)
• DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations [119.1953397679783]
  We focus on advancing the state-of-the-art in interpreting multimodal models. Our proposed approach, DIME, enables accurate and fine-grained analysis of multimodal models.
  arXiv  Detail & Related papers  (2022-03-03T20:52:47Z)
• Knowledge-informed Molecular Learning: A Survey on Paradigm Transfer [20.893861195128643]
  Machine learning, notably deep learning, has significantly propelled molecular investigations within the biochemical sphere. Traditionally, modeling for such research has centered around a handful of paradigms. To enhance the generation and decipherability of purely data-driven models, scholars have integrated biochemical domain knowledge into these molecular study models.
  arXiv  Detail & Related papers  (2022-02-17T06:18:02Z)
• Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models [55.41644538483948]
  In modern data science, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. We use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm's desired properties.
  arXiv  Detail & Related papers  (2021-07-07T11:17:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.